- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Markets of Minds allocates scarce expert assistance by predicting its benefit. In a 240-case matched replay at 30% assistance, it achieves 85.42% persona adherence versus 82.92% for the strongest baseline.
Critical strengths
-
Distinguishing task difficulty from assistance benefit is useful.
-
Shared cases, cached outcomes, and quotas isolate recipient selection.
-
Multimodal, population-allocation, and computational scaling experiments broaden the evaluation; Table 9 reaches 100,000 agents.
Critical weaknesses
-
Table 4 needs reconciliation. The stated replay implies 156 default successes and 49 repairs under Ours. With 15 missed repairs, 64 cases are repairable. Threshold and FCFS miss 45 and 52, implying success ceilings of 72.9% and 70.0% before harmful help, yet report 78.10% and 77.50%. Clarify aggregation and case accounting.
-
The primary implementation predicts repairs without separately estimating regression risk. Table 8 includes harmful-help outcomes, but a matched full-estimator versus repair-only ablation is needed to establish the signed formulation's added value.
-
Table 3's 82.66 matches the multimodal average, but its Survey development protocol and target evaluation require clarification. Persona adherence also lacks reported human or second-judge validation.
Questions for authors
-
Can you reconcile Table 4 using underlying success, repair, and regression counts?
-
Does the full signed estimator improve over repair-only scoring when harmful help occurs?
-
Which task does Table 3 ablate, and how reliable are the persona-adherence labels?