acceptodds
Under review as a conference paper at ICLR 2027

Decentralized LLM Workflows via Sequential Auctions

Abstract

We present a scalable framework for solving multi-step tasks through a decentralized market of self-interested agents. Prior theoretical work demonstrated that agents organized through sequential auctions can converge to a globally optimal policy in a Markov decision process (MDP), but relied on the restrictive assumption of the existence of perfect clones as a model of perfect competition. In reality, exact clones drive agent profit to zero, violating the natural risk-reward incentives of participants. Conversely, we empirically demonstrate that the opposite extreme, absence of competition, also leads to market failure and value extraction collapse. Between these two extremes, we investigate how heterogeneous agents can survive in markets that span the critical middle ground of *imperfect competition*. To be profitable, agents learn to strategically shade bids to secure positive individual profit margins, while still ensuring that the marketplace functions effectively enough so that collaborative tasks posed by market consumers can still be solved. We formulate and evaluate four bidding algorithms (Mirror Descent, Thompson Sampling, Bayes-Optimal, and PPO) across three auction protocols (1st-Price, 2nd-Price, and Credit-Conserving Vickrey). Across a diverse set of domains, we demonstrate that imperfect competition successfully sustains necessary agent profit margins while maintaining high task success rates at low costs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-Agent, Market, Auction, LLM

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_TKwaOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a decentralized LLM workflow framework based on sequential auctions, where heterogeneous agents learn bidding strategies to coordinate multi-step tasks while maximizing individual profits. The authors investigate imperfect competition as an alternative to the perfect competition assumption in prior work and evaluate several bidding algorithms. Experiments demonstrate promising task performance and positive agent profits.

Critical strengths
  1. The proposed approach combines sequential auctions with lightweight learned bidding policies while keeping the underlying LLMs frozen, offering a relatively simple and computationally efficient way to coordinate multiple agents.

  2. The experiments cover multiple domains, including information retrieval and mathematical reasoning, demonstrating competitive task performance while maintaining positive agent profits under imperfect competition.

Critical weaknesses
  1. The proposed framework builds upon existing sequential auction-based decentralized reinforcement learning methods, extending them to heterogeneous LLM agents with learned bid-shading strategies. While these extensions are practically meaningful, the bidding algorithms primarily adapt established techniques, and further theoretical analysis would help strengthen the methodological contribution.

  2. The LLM experiments rely on a centralized assessor that provides agent-specific relevance scores, raising concerns about how much of the routing performance actually comes from decentralized bidding. Moreover, the lack of a centralized learned router using the same features and computational budget makes it difficult to isolate the benefits of the auction mechanism.

  3. The claim that imperfect competition enables sustainable agent economies is primarily supported by controlled experiments with fixed agent populations. The paper does not sufficiently investigate market stability under dynamic participation, strategic manipulation, or varying competition levels, limiting the generality of its central conclusion.

Questions for authors
  1. Since the LLM experiments rely on a centralized assessor to provide agent-specific relevance scores, how can the authors disentangle the contribution of decentralized auctions from that of the assessor? Would a centralized learned router using the same features achieve comparable performance?

  2. How does strategic bid-shading affect global task efficiency? Since agents optimize individual profits rather than overall task performance, could bid-shading lead to suboptimal agent selection? Can the authors provide an ablation comparing learned bid-shading with truthful value-based bidding?

Reviewer_KCqDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper looks at how to orchestrate decentralized LLM workflows using sequential spot markets. In this setup, self-interested agents bid for, modify, and resell task states to pass credit back upstream. Instead of relying on the unrealistic "perfect-clone" assumption from previous work, the authors focus on imperfect competition. They examine how different agents learn to strategically adjust their bids (bid shading) to actually make a profit. The paper tests four bidding algorithms across three auction setups, using synthetic translation and LLM tasks like News Market, HotpotQA, and Omni-MATH. The results show that a lack of competition can cause the market to crash, but learned bidding in imperfect markets can successfully support multi-step workflows. Still, relying on a centralized assessor and the inconsistent accuracy-cost trade-offs are significant limitations.

Critical strengths
  1. The paper tackles a real issue with earlier sequential auction models: agents actually need to make a profit to participate. Moving from the idealized "perfect clones" idea to imperfect competition makes the scenario much more practical for open markets run by independent providers.

  2. The synthetic translation experiments do a great job showing why competition is key. By showing how a lack of competing buyers starves upstream agents and breaks the workflow, the paper gives solid proof of when the market works and when it falls apart.

  3. The experiments show real promise for using sequential auctions in multi-step workflows. The accuracy boost over the ReAct baseline on News Market, and over standard refinement strategies on Omni-MATH, proves this approach can work outside of just synthetic setups.

Critical weaknesses
  1. It's hard to tell how much of the performance boost comes from the auctions themselves versus the centralized assessor. Since the assessor already scores which agents are useful and gives specific prompts in News Market (like when to get evidence vs. when to answer), it does a lot of the heavy lifting. We really need a baseline that just picks agents based on those scores to see what learned bidding actually adds. Because of this, the claim about "decentralized coordination" feels a bit weak right now.

  2. The cost advantage isn't very clear across the board. For example, if you include the assessor, News Market costs about three times as much as ReAct (even though it's more accurate). On HotpotQA, it gets similar accuracy but costs eight times more. For Omni-MATH, cost is only measured by refinement counts, which misses the full cost of verification and assessment. Altogether, it's hard to say if the accuracy gains are worth the extra cost.

  3. The argument that agents will want to participate needs more backing. Just showing positive profits for a fixed group of agents doesn't prove they’d actually join or stay once all costs are factored in. It’s also unclear how platform profit is calculated, does it include the costs of the assessor and training? Plus, how would the evaluator-provided terminal rewards translate to actual customer payments in the real world? Since economic viability is a main selling point of the paper, these gaps need addressing.

  4. I'm not sure if first-price auctions will hold up for longer workflows. Since each winning agent lowers its bid to make a profit, the revenue passed upstream drops at every step. The LLM tasks tested here are pretty short (only 2-3 steps on average), so it’s unclear if this bid-shading would end up starving upstream agents in longer, deeper pipelines.

Questions for authors
  1. How does the auction setup compare to a simple baseline that just routes tasks to the highest-scoring agent (based on the centralized assessor)? This would really help show if learned bidding is actually better than just using the assessor's built-in routing rules.

  2. What is the total token/API cost for Omni-MATH across all stages (generation, verification, refinement, and assessment) compared to the baselines? Refinement counts alone don't give the full picture of whether the accuracy improvements are truly cost-effective.

  3. Could you clarify exactly how platform profit is calculated under first-price auctions? What specific revenues and expenses are included in the cumulative profits? For instance, do they include the assessor's overhead and the costs of training the bidding policies? Also, how would the evaluator-provided terminal rewards translate to actual customer payments in a deployed market?

  4. You mentioned that success rates on News Market and HotpotQA are measured over the final 100 episodes. Are these using held-out task instances, and do the bidding networks keep updating during this time? Also, are the cost and step-count metrics in Tables 1 and 3 based on these exact same episodes?

Reviewer_aXhjOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes a framework where the LLMs bid on tasks through auctions. The task can be decomposed sequentially such that, in every stage, the LLMs bid for the tasks, process the tasks, and then release the tasks to the next-stage auction and receive income from the auction. Such a procedure has been theoretically shown to converge to a Nash Equilibrium under certain unrealistic assumptions. This work extends the prior theoretical results into a more practical imperfect competition setup where the agents have enough incentives to participate in the auctions. This work empirically shows that such an auction mechanism can improve the efficiency of task allocation and effectively reduce the total costs of production.

Critical strengths
  1. This work is well motivated by theoretical works, and the proposed framework is practical. In particular, participation incentives matter when agents belong to independent providers. Studying whether useful workflows survive between perfect competition and monopsony is a worthwhile extension of earlier work.

  2. The empirical results shown are promising.

  3. This paper is very well-written.

Critical weaknesses
  1. It seems like the agents must estimate expected downstream revenue and execution costs before performing a task. Although the framework learns valuations from historical payoffs, it does not systematically evaluate their accuracy or the trade-off between estimation accuracy and assessment overhead. Noisy or biased estimates could distort bids, select inefficient agents, and undermine profitability and backward credit propagation. It therefore remains unclear whether the reported market viability persists under realistic valuation errors, particularly for unfamiliar tasks or changing market conditions.

  2. The centralized assessor evaluates every candidate’s usefulness and supplies the features used for bidding. It is unclear whether this assessor already provides enough centralized information to the LLM such that other mechanisms, together with the assessor's advice, can perform better than the proposed auction framework. The paper needs to show what the auctions accomplish beyond the routing intelligence already supplied by the assessor.

Questions for authors
  1. How accurately can agents estimate task valuations before execution? How do noisy or biased estimates affect market convergence and equilibrium quality, measured by task performance, social welfare, and agent profitability?

  2. Could you provide more insights into what the auctions accomplish beyond the routing intelligence already supplied by the assessor?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.