acceptodds
Withdrawn from ICLR 2027

Markets of Minds: A 51-Billion-State Economy of Intelligence

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

When intelligence is shared, reasoning becomes a constrained allocation problem. In a population of agents with limited access to expert models, an intervention can benefit its recipient yet still be a poor use of shared capacity if another feasible intervention has greater expected value. Assistance must therefore be evaluated through both its marginal benefit and the opportunity cost induced by budgets, access, scarcity, and service capacity. To this end, we introduce Markets of Minds, a framework for allocating expert reasoning across agents before expert outputs are observed. For each feasible agent–service pair, the framework estimates the expected gain from assistance relative to the agent's default continuation, explicitly separating repair benefit from regression risk, and jointly clears assistance contracts under shared resource constraints. We then study this problem in a dependency-aware agent economy with 51.2 billion addressable persona–market states spanning capabilities, cognitive budgets, provider access, scarcity, and network roles. These dependencies couple allocation feasibility across agents rather than the outcome of a fixed contract. Empirically, in a matched replay of 240 cases under a common 30% assistance quota, the frozen allocator reaches 85.42% task-level persona adherence versus 73.86% for equal-capacity random allocation, a gain of 11.56 percentage points. The results show that assistance value is heterogeneous and can be negative, and that scarcity changes which interventions are worth purchasing. Together, these findings motivate a population-level view of intelligence where the value of assistance depends not only on what it can achieve, but also on which alternative uses of scarce cognitive capacity it displaces.

Primary area:
optimization
Keywords:
AI markets, market design, resource allocation, multi-agent systems, economics of AI, cognitive markets, population-level allocation, LLM agents

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_LbSYOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Markets of Minds allocates scarce expert assistance by predicting its benefit. In a 240-case matched replay at 30% assistance, it achieves 85.42% persona adherence versus 82.92% for the strongest baseline.

Critical strengths
  1. Distinguishing task difficulty from assistance benefit is useful.

  2. Shared cases, cached outcomes, and quotas isolate recipient selection.

  3. Multimodal, population-allocation, and computational scaling experiments broaden the evaluation; Table 9 reaches 100,000 agents.

Critical weaknesses
  1. Table 4 needs reconciliation. The stated replay implies 156 default successes and 49 repairs under Ours. With 15 missed repairs, 64 cases are repairable. Threshold and FCFS miss 45 and 52, implying success ceilings of 72.9% and 70.0% before harmful help, yet report 78.10% and 77.50%. Clarify aggregation and case accounting.

  2. The primary implementation predicts repairs without separately estimating regression risk. Table 8 includes harmful-help outcomes, but a matched full-estimator versus repair-only ablation is needed to establish the signed formulation's added value.

  3. Table 3's 82.66 matches the multimodal average, but its Survey development protocol and target evaluation require clarification. Persona adherence also lacks reported human or second-judge validation.

Questions for authors
  1. Can you reconcile Table 4 using underlying success, repair, and regression counts?

  2. Does the full signed estimator improve over repair-only scoring when harmful help occurs?

  3. Which task does Table 3 ablate, and how reliable are the persona-adherence labels?

Reviewer_nEAKOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper introduces a new framework for allocating limited AI expert services across an agent population. In the paper's theoretical set up there are nn agents. Each agent ii has an observable state sis_{i} describing its task, execution history and available resources. An agent's task is binary. That is an agent can either succeed or fail at a task. Each agent can be assigned one of mm services to assist them in completing their task. Each service jj can be assigned to at most one agent. A service can increase the probability that an agent will succeed but can also reduce it. The authors then consider the problem of optimally allocating services so that the expected number of successfully completed tasks is maximised and show that the contribution Δij\Delta_{ij} of a given service to a task's success depends on three terms:

  • Default success: The agent's intrinsic probability of completing the task without service assistance.
  • Regression Risk: The probability that an agent would fail the task with service assistance given that it would succeed without assistance.
  • Repairability: The probability that an agent would succeed with service assistance, given that it would fail without assistance.

The authors then provide a lower bound on the error of an allocation if the contribution of each service to each agent's success has uniform error.

Motivated by this theoretical set up, the authors then propose a practical framework for service allocation called Market of Minds (MoM). MoM receives demand packets from agents and estimates the contribution of each service to each agent's success, by estimating the default success, regression risk and repairability. Using these estimates an optimal allocation is then computed and used to allocate services to agents. MoM is evaluated in a host of empirical settings against multiple methodologies adapted from other work.

Critical strengths

I think the central problem is interesting and relevant, especially as AI adoption increases throughout society. There has obviously been a lot of work and care taken to develop a wide battery of personas and tasks to exhaustively test the proposed framework.

Critical weaknesses

The theoretical contributions of the paper are weak. Both of the stated theoretical results are extremely obvious and almost immediate from definitions. Whilst the decomposition of Δij\Delta_{ij} into three terms may be intuitive it is not necessarily useful for solving the allocation problem more efficiently. Meanwhile the second proposition is not used or exploited anywhere in the paper.

The implementation of MoM used for experiments is also inconsistent with the formal specification of MoM, as pointed out by the authors themselves on line 241. In particular, repairability, default success and regression risk are not estimated independently and combined. This further brings into question why this decomposition forms such a central role in the paper's narrative. In fact, the MoM implementation used in experiments does not even compute Δij\Delta_{ij} or compute an optimal allocation. It is thus unclear why any of the theoretical set up discussed in Section 2 is relevant.

The presentation of empirical results is also concerning. Many of the plots have an incredibly small x-axis range suggesting that the performance of MoM relative to other benchmarks is more significant than the modest improvements observed in reality. Moreover, there are no confidence intervals in any of the plots or tables, making it difficult to know whether the performance benefits of MoM hold reliably.

The written presentation of the paper also falls short. There is excessive use of text colouring that has no clear pattern and only serves to distract the reader. Figure 2 did not help me understand the paper's contributions at all. There are arrows flowing across the diagram in every direction and it seems unreasonable to have such a small caption for such a complicated image. Figure 1 was also distracting and did not help me understand the paper's contributions. More generally, many aspects of MoM are still unclear upon multiple readings. There is a lot of confusing terminology that is never explained. For example, line 250 says "Matched replay uses the same trajectories for every policy...". What is a "trajectory"? Likewise what is the role/function of the "actor" and the "judge" (the first mention of these terms outside of the introduction is in the experiments section)? What is a "matched replay" and an "agent case"? These are basic terms that are not defined for the reader anywhere in the main body. There are also a lot of sentences that read very "AI generated". For example, the first sentence on line 42 is vague at best.

Questions for authors

I strongly encourage the authors to prioritise a revised version of this work for another venue. Personally I think the theoretical justification at the beginning of the paper is completely unnecessary and that for this kind of problem the empirical study is what really counts. With this observation in mind, I would cut the theoretical results from the paper completely, and use the additional space to provide more clarity about how MoM is practically implemented. For example, just clarifying in the main body that the judge is a language model that evaluates task success would help readers understand the setup better. Likewise being more consistent with language would hugely improve the paper. For example, is there a difference between an agent case and a demand profile, if there is, what is it? I would also rerun the empirical experiments to at least provide confidence intervals.

At the moment I have scored the paper as a clear reject for the following reasons:

  • The connection between theoretical results and the framework proposed are tenuous at best.
  • The written presentation is poor. It is difficult to understand the framework being proposed and how it differs from existing methods.
  • The presentation of empirical results in the main body include no confidence intervals/error bars.
  • The empirical improvements offered by MoM is relatively modest.
  • Figures are distracting and did not improve my understanding.
Reviewer_xGUjOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper is a methodology paper addressing the issue of AI routing, i.e. how to direct tasks to other stronger models that have better expertise than the base model, and which expert to allocate this to etc.

The authors formulate this problem into an allocation problem through "expected assistance gain", which measures how much help an intervention may contribute (e.g. repairing a failed response), under shared response. The main contribution of "Market of Minds" is a framework on how to estimate this gain before outputs were observed. This is done primarily via developing a persona-market state space across budget, expertise, and other variables that affects the potential value of an intervention. Experiments include a matched replay of 240 agent tasks, multimodal benchmark experiments, and shared-capacity allocation studies.

Critical strengths

Problem formulation - The paper highlights an interesting resource-allocation problem re model routing

Critical weaknesses
  1. Relationship between framework and evaluated mechanism is unclear. The proposed framework included several possible levers / mechanisms for (e.g. heterogeneous services, budgets, quality of intervention). However, the matched experiment uses one service, fixed per-environment quotas, and a repair-probability score. Under these conditions, allocation is possibly reduced to ranking requests and selecting the top ones? I can't exactly identify which results demonstrate the broader market mechanism instead of an improved ranking model in the matched experiment. Similarly, the role of the persona–market economy in the reported gains is not sufficiently clear. While the authors describe a large state space of possible persona, the paper does not establish how these dimensions affect allocation or downstream performance much.

  2. The results lack clarity, and overall not fully supportive of the authors' claims. Table 2 reports three-run means without error intervals, while several differences between routing methods are relatively small. Furthermore, the assistance pool contains models whose standalone average performance is below that of the base model. This is odd, because one would intuitively expert the experts be at least better at, if not entirely dominate, the base model (why route otherwise?). The highest-scoring method also uses the fewest expert calls, leaving unclear how much of its advantage comes from choosing better assistance v.s. just not calling any experts entirely

Questions for authors
  1. For the matched experiment, my understanding is that it uses a pre-determined repair-probability score for each expert, followed by top-24 selection within each environment, with most of the levers fixed? (n.b. Appendix B also states that this is equivalent to local top-K ranking?) In that case, were there any demonstrable benefits in the factors outlined in your allocation mechanism, i.e. budgets, provider access, or multiple service capacities etc., rather than from a better value predictor?

  2. Could you clarify which estimator and allocation rule are used in each experiment, particularly the multimodal and 512-agent studies? In the multimodal experiment, does the allocator jointly select the request and expert model? Where do the persona–market attributes enter these decisions?

  3. Could you provide paired confidence intervals for the main comparisons, especially against Agora in Table 1 and the strongest routers in Table 2? For the multimodal results, it would also help to compare methods at matched realized expert use and report how many interventions repair failures versus regress correct answers.

  4. Could you reconcile the results in Table 4 with the stated 240-case matched replay? Pooled top-72 has fewer missed repairs than "Ours" but lower overall success, despite very little reported harmful help. Several success rates also do not appear to correspond to integer counts out of 240. Are these averages over additional runs, or is a different scoring protocol used?

  5. Why were metrics such as Jain fairness used? It is not defined anywhere, but it seems to be about how well resource is distributed across participants? why?

  6. Can you clarify / break down the components in API-equivalent cost calculation? were these costs are measured during deployment or reconstructed from the cached response pool?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.