- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
NanoMA lets agents create, stop, query, and message other agents during task execution, so the team structure can change as work progresses.
Critical strengths
NanoMA reaches high all-pass on RepoZero, much above the baseline.
Critical weaknesses
-
It changes queries and message recipients, but does not isolate creating or stopping agents. Better communication could help even with a fixed team. The ablations does not establish the benefit of changing team membership. Compare with fixed teams using the same messaging, queries, and runtime.
-
Table 7 reports 3.53M tokens for NanoMA versus 0.43M for CrewAI. Extra attempts and checks could explain part of the gain. The current advantage are difficult to attribute to coordination alone. Compare success at equal token budgets and report cost per successful task.
-
It adds validation, rollback, a durable result ledger, and publication checks alongside flexible teams. These features could improve results without changing team membership. This bundled design make the source of NanoMA’s gain unclear. A fixed-team control should retain the same ledger, workspace rules, and validation.
-
It counts send calls, but one broadcast can expose many agents to the same text. Similar call rates therefore do not mean equal received context. The current comparison does not separate recipient choice from extra distracting messages. This distinction need received-token counts and comparisons with similar message volume and team size.
-
It allows later overlapping edits to overwrite earlier changes. Rollback occurs when registered checks detect failure, so a lost requirement could survive if those checks miss it. The paper does not measure this failure rate. The current validation process do not establish that parallel work is preserved. Test overlapping patches with known requirements and report overwritten successes and recovery.
Questions for authors
Please refer to the weaknesses.
21 comments
W1-b: Prescribed Topologies with Default Communication
W1-b: Prescribed Topologies with Default Communication
To address your request for team comparisons with the same messaging, queries, and runtime, we write skills that constrain spawn-tool calls so agents can only be created according to the prescribed star, transitive-tournament, or strict-binary-tree structure, while retaining default query/send permissions, the model, runtime, and resource limits. These controls preserve inter-agent communication while constraining team organization. Their All-pass rates are 51.80%, 55.40%, and 51.08%, and Micro scores are 79.43%, 80.80%, and 78.32%, respectively, versus 66.91% All-pass and 93.32% Micro for autonomous NanoMA. Actual team sizes and token use are reported below.
Prescribed orchestration with default communication
| Orchestration | All-pass | Micro | Total tokens | Mean agents |
|---|---|---|---|---|
| Default NanoMA · autonomous | 93/139 (66.91%) | 93.32% | 493.69M | 6.82 |
| Prescribed star · centralized | 72/139 (51.80%) | 79.43% | 197.04M | 3.51 |
| Prescribed transitive tournament | 77/139 (55.40%) | 80.80% | 267.43M | 4.17 |
| Strict binary tree · local | 71/139 (51.08%) | 78.32% | 218.01M | 5.01 |
Each prescribed topology is implemented through skills that constrain spawn-tool calls so that agent creation follows the specified structure. All configurations retain default query/send permissions and use the same backbone, runtime, and resource limits. All-pass uses 139 tasks, counting missing runs as failures. Micro uses the same aggregation as the default result. Topology definitions are provided in the main text and appendix. Actual token use and team sizes are reported, not assumed equal.
W1-c: Spawn Ablation
W1-c: Spawn Ablation
To address your question about agent creation, we compare autonomous spawning, forced spawning at every task_create, and single-agent execution with spawn disabled on shorter Terminal-Bench 2 tasks and longer EdgeBench tasks. Terminal-Bench 2 tests whether selective delegation also benefits shorter tasks that may require only simple collaboration. Within each benchmark, the model, runtime, and budget settings are identical. Autonomous spawning scores highest on both benchmarks, while forced expansion performs worse than single-agent execution, supporting the value of deciding whether and when to delegate.
Agent-creation policy ablation
| Configuration | Spawn policy | Terminal-Bench 2 · Luna xhigh | EdgeBench |
|---|---|---|---|
| Default NanoMA | Autonomous decisions | 57.30% (51/89) | 22.10 |
| Forced spawning | Spawn at every task_create | 41.57% (37/89) | 16.29 |
| Spawn disabled | Single agent | 53.93% (48/89) | 18.21 |
*Within each benchmark, all three configurations use the same backbone, runtime, and budget settings. EdgeBench values are scores, not percentages.
W1-a: Spawn and Inter-Agent Communication
Thank you for your suggestions on separating orchestration and communication effects.
W1-a: Spawn and Inter-Agent Communication
To address your question about separating agent creation from communication, completely disabling spawn in NanoMA’s single-root initialization leaves only one agent and also removes inter-agent communication. The no-spawn ablation therefore changes both team formation and the opportunity to communicate; removing this tool alone cannot independently measure their contributions. We consequently use prescribed-topology controls that retain multiple agents and default communication capabilities to examine the effect of team organization.