Semantic Relaxation with Large Language Models

Recently, agentic workflows have achieved substantial advances across downstream tasks by enabling multiple Large Language Models to interact with external environments (\eg, tools, memory, and API). Nevertheless, their discrete and non-differentiable nature poses a fundamental optimization barrier in that external feedback often fails to translate into explicit update directions. In this work, we carefully design SemanticRelaxation, a domain-agnostic optimization framework that semantically bridges discrete generation with continuous optimization. Specifically, SemanticRelaxation integrates two core components: (i) Semantic Representation Learning (SRL), which learns a joint embedding space by promoting cross-modal correlation between descriptions and their corresponding artifacts while preserving semantic structure; and (ii) Barycentric Semantic Optimization (BSO), which explicitly guides semantic directions through barycentric aggregation and accumulates historical updates as trajectory momentum after parallel transport. Finally, SemanticRelaxation effectively encourages agentic workflows by grounding executable artifact refinements in retrieved semantic trajectories within an iterative optimization loop. Our strategy achieves state-of-the-art performance on benchmarks spanning Agentic Workflows, Code Generation, Molecular Editing, and Neural Architecture Search, increasing the success rate on M3Tool by up to 12.2$\%$ and reducing token consumption on HumanEval by 29.2$\%$. Comprehensive empirical analyses further support the effectiveness and validity of our framework.