acceptodds
Under review as a conference paper at ICLR 2027

PRISM: PRIvileged Sibling-guided Multi-Agent Topology Design

Abstract

The effectiveness of large language model multi-agent systems depends on how agents are selected and connected for each query. Autoregressive topology designers learn these decisions from execution-verified query-graph pairs, but continued supervised training does not necessarily improve downstream performance. In our experiments, continued supervised fine-tuning increases output concentration, with uneven performance across benchmarks. We propose PRISM, which reuses the verified corpus through on-policy self-distillation from a privileged teacher. During training, the teacher receives a different graph that successfully solved the same query through a low-rank conditioning branch. The student samples topologies under its own policy and learns from the teacher's distributions on these trajectories, together with a supervised anchor on verified graphs. The teacher is discarded at deployment, leaving the designer architecture unchanged, and post-training requires no additional LLM executions. Across six reasoning and code benchmarks with Qwen2.5-7B-Instruct, PRISM outperforms the evaluated topology-design baselines. Matched teacher controls support the contribution of query-associated privileged information, while output-distribution measurements show reduced concentration relative to continued supervised fine-tuning.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multi Agent System, Post-training Method, On-Policy Self-Distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_tEH3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PRISM, a post-training method for multi-agent topology generation. It trains a teacher conditioned on another successful topology of the same query as privileged information, and then performs on-policy distillation (OPD) to improve the student topology generator. Experiments show improvements over SFT and existing topology design baselines.

Critical strengths
  1. The paper studies an interesting problem of learning multi-agent topology generators from execution-verified graphs.
  2. The experiments cover six benchmarks and include useful controls to distinguish the contributions of OPD and privileged information.
Critical weaknesses
  1. Mismatch between the claimed challenges and the proposed solution. The paper motivates the method by the lack of preferences among successful topologies, distribution bias in SFT, and exposure bias. However, another successful topology does not provide any preference or quality signal over the training target. OPD addresses exposure bias, but it is unclear how the proposed method resolves the first two challenges.
  2. Limited novelty in methodology design. The proposed approach mainly combines existing on-policy distillation with privileged information learning. It is fine, but during doing so, the motivation for using another successful topology as privileged information is not sufficiently justified.
  3. Insufficient evidence for improved query-specific topology design. The topology intervention experiments in Appendix H show that PRISM's role assignments perform similarly to random or query-permuted assignments. Moreover, on AQuA, the method mainly recovers the performance degradation introduced by SFT, with only a marginal improvement over the original ARG-Designer.
Questions for authors

Why should conditioning the teacher on another successful topology provide better supervision? Since both topologies are successful, what information allows the teacher to distinguish better and worse topology decisions?

2 comments
AuthorsOpenReview ↗

Official response for Reviewer tEH3

Thank you to Reviewer tEH3 for the thoughtful comments. We address the questions and concerns below.


[Q1] What does the sibling give the teacher, if both topologies succeeded?

The sibling does not let the teacher rank topologies. The teacher is fit by likelihood on execution-verified graphs, conditioned on a different verified graph for the same query. It therefore models the family of topologies that worked for that query, and the student is pulled toward this conditional distribution rather than the marginal over all queries. Two controls show that the content is used and survives distillation:

  • With a real sibling, the teacher's held-out log-likelihood improves by 2.45 nats; with another query's sibling, the improvement is zero (Section 6.1).
  • At matched teacher epochs, rollouts and loss, PRISM beats the constant-input teacher on all six benchmarks (+1.66 on average, sign test p = 0.031; Section 6.2).

[W1] The claimed challenges do not match the solution.

We agree and will revise the framing. On-policy distillation addresses exposure bias (the constant-input teacher improves AQuA by +9.05 over SFT). Conditioning on a query-specific sibling addresses SFT's distribution collapse: the top-1 topology share is 0.346 for SFT, against 0.171 for the starting checkpoint and 0.157 for PRISM (Section 5.3). The lack of a preference among successful graphs is not resolved by PRISM, and the Introduction will say so.


[W2] Novelty

We agree that on-policy distillation and privileged information are established ideas, and we will describe PRISM as an adaptation of them rather than a new learning principle. Our contribution is their combination in a setting where neither transfers directly: graph-valued actions, a verified graph for the same query as the privileged signal, and no reward model or preference data. To the best of our knowledge, this is the first work to study post-training methods for multi-agent system topology design. Prior topology generators are trained by supervised learning or reinforcement learning on the generator itself, and none addresses the post-training problem that PRISM targets (exposure bias and distribution collapse in graph generation). We will add a related-work paragraph that states this positioning and the differences from existing approaches.


[W3] Evidence for query-specific design, and AQuA

We do not claim query-specific role assignment, and Appendix H already states that the intervention "does not establish that PRISM's particular crews, or their assignment to particular queries, carry benefit." Consistent with this, PRISM's crews permuted across questions score 78.39 against 77.66 for the original assignment (p = 1.00), and a size-matched random crew scores 75.46 (p = 0.61). What the intervention does show is that SFT's own crew is worse (61.54 against 77.66; 4 vs 21 discordant, p = 0.0009), so PRISM's gain comes from avoiding SFT's collapse.

On AQuA we agree that the gain over ARG-Designer is small (76.51 vs 75.33). The larger gains over ARG-Designer are on GSM8K (87.77 vs 80.21) and in the four-benchmark average (89.16 vs 86.70).

We hope our responses address your concern, and we will add these discussions to the revision. If any concerns remain, we would be happy to discuss them further.

Dear Reviewer tEH3,

We sincerely appreciate the time and effort you devoted to reviewing our work, as well as your constructive and encouraging comments. We have addressed each of your concerns in our rebuttal and hope our responses are clear and satisfactory. Should you have any further questions or suggestions, we would be glad to discuss them in more detail.

Thank you for your thoughtful engagement with our paper.

Sincerely,

The Authors of Submission 12333

Reviewer_cgLjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PRISM, a post-training method for an existing multi-agent topology generator. It first trains a privileged teacher that receives another successful topology for the same query. The student then learns the teacher’s decision distributions along its own generated topology trajectories, while retaining supervised training on verified graphs. Only the student is deployed, leaving the original generator architecture unchanged. The method is evaluated on six benchmarks, with all task-executing agents using Qwen2.5-7B-Instruct.

Critical strengths

S1. The method makes additional use of existing successful executions without requiring new multi-agent executions during post-training.

S2. The reported improvements have meaningful experimental support. The methods are reevaluated under a common execution stack, and the ARG-Designer baseline includes its complete training procedure. The three-seed results in Appendix D show higher mean accuracy than ARG-Designer on all six benchmarks. The authors also explicitly acknowledge that the advantage over SFT on MultiArith is not consistent. These checks strengthen the credibility of the main results.

Critical weaknesses

W1. The added value of learning query-dependent topologies over simpler collaboration strategies remains insufficiently established.

The intervention experiments in Appendix H are particularly relevant. On 91 AQuA questions selected according to SFT’s outputs, with chain wiring imposed across conditions, PRISM’s original crews, cross-query-permuted crews, and size-matched random crews achieve mean accuracies of 77.66%, 78.39%, and 75.46%, respectively. The corresponding tests do not detect a significant advantage for PRISM over the latter two conditions. Because the question set is selectively constructed and wiring is fixed, this does not establish that random strategies can generally replace PRISM. However, it also does not rule out a simpler explanation: a substantial part of the improvement comes from avoiding poor role compositions learned by SFT, rather than more effectively matching queries to collaboration structures.

The conclusion appropriately acknowledges that the benefit of query-specific assignment has not been established. Nevertheless, this issue concerns not only the mechanistic explanation, but also the practical value of a learned planner relative to simpler alternatives. Comparisons against fixed, random, or cross-query-permuted strategies on a representative question set selected independently of SFT’s behavior, with matched execution protocols and budgets, would be more convincing. A constant-input teacher during training does not replace these deployment-policy controls.

More broadly, although the paper includes a vanilla single-agent baseline, PRISM uses approximately 4.04 LLM calls per query on average. That comparison does not establish whether learning a collaboration graph is preferable to allocating a comparable inference budget to repeated attempts or checking by a single agent. This does not require training a new, stronger single agent, but it does call for a clearer distinction between the benefits of additional inference computation and those of learned collaboration structure.

W2. The trained planner’s dependence on the agent configuration needs to be defined more clearly.

All agents in the experiments use the same underlying language model, and supervision comes from topologies retained after successful execution. The evidence therefore supports post-training effectiveness under the evaluated agent configuration; it does not establish effectiveness after that configuration changes. Although the role-encoding mechanism in Section 3 structurally supports an extensible role pool, accepting a new role as input is not equivalent to organizing that role effectively.

Please clarify whether adding or removing roles, changing role prompts, or replacing the underlying language model requires collecting new successful graphs and retraining the planner. If so, is the intended deployment setting primarily a relatively stable agent configuration with reusable execution records? (Although the authors already identify cross-backbone transfer as future work.)

Questions for authors

Q1. Which properties of the agents are assumed to remain unchanged after the planner is trained? How should the planner be adapted when the role pool, role prompts, or underlying language model changes, and is there existing evidence supporting that adaptation procedure?

Q2. Are there existing budget-matched comparisons against simple fixed/random collaboration strategies on a question set selected independently of SFT’s outputs? Do these results establish an additional benefit from learning query-dependent topologies?

Q3. Starting from the same checkpoint, does the AQuA regression persist when continued supervised training retains the objective in Eq. (2)? How does PRISM compare with that baseline in performance and additional training cost?

3 comments
AuthorsOpenReview ↗

Official response for Reviewer cgLj

Thank you to Reviewer cgLj for the detailed review. We address each question and concern below.


[Q1, W2] Dependence on the agent configuration

All agents share the same backbone, role pool and role prompts, and the supervision consists of graphs that succeeded under that configuration. Our evidence therefore supports post-training under a fixed configuration. Changing the role pool, role prompts or backbone changes which graphs succeed, so supervision would have to be re-collected and the planner retrained. We have no experiment supporting an adaptation procedure for that case. The role encoding in Section 3 lets the planner accept a new role as input, but we do not claim it organises a role it has not seen.


[Q2, W1] Learned query-dependent topologies versus simpler strategies

We ran the requested controls on all 254 AQuA test questions with a matched execution protocol: chain wiring, one round, the same decision method and backbone, and three executions per condition.

ConditionAccuracyPRISM vs. condition (McNemar p, majority verdict)
PRISM crew assignment71.92%-
Same crews, permuted across questions63.65%0.0003
Size-matched random crew62.34%0.0005
One fixed crew (PRISM's most common)56.69%< 0.0001

Permuted vs. random crews: p = 0.39 (indistinguishable).

PRISM's assignment is significantly better than each control. Permuted and random crews perform alike, so the gain comes from matching crews to questions rather than from the crews themselves, and a single fixed crew, even PRISM's most common one, does not recover it.

These results come from one benchmark and one seed, use chain wiring, and draw the random baseline uniformly from the role vocabulary at PRISM's crew sizes.


[Q3] Does the AQuA regression persist under continued supervised training, and at what cost?

Yes. Starting from the same checkpoint, continued training with the Eq. (2) objective leaves AQuA below ARG-Designer in every setting we ran (seeds 42-44):

AQuAAccuracy
ARG-Designer (Table 8)75.33
SFT, early-stopped (Table 8)62.73
SFT-30, last epoch67.32
SFT-70, last epoch68.50
PRISM (Table 8)76.51

SFT-30 and SFT-70 use the same number of epochs as PRISM's student stage and as its teacher plus student stages, respectively. On one NVIDIA A5000, SFT takes 2.1 min (30 epochs) and 4.9 min (70), against about 10.6 min for PRISM (3.2 teacher + 7.4 student). At inference, PRISM uses more tokens on GSM8K (4,531 per question vs 3,808-4,188 for the SFT arms) and about the same on AQuA (3,296 vs 3,174-3,334).


[Inference budget]

The paper compared PRISM (about 3.9-4.0 LLM calls per query) only against a one-call single agent. We therefore added a budget-matched baseline: one "Math Solver" agent (the prompt used by the graph framework itself, with no graph and no designer) answers each question N times at temperature 0.7, and the majority vote is scored. Same Qwen2.5-7B-Instruct, full AQuA test set, 3 repeats for N > 1:

MethodAccuracyCalls / queryTokens / query
Single agent, N=161.811568
Single agent, N=4 votes62.60 ± 1.7242,251
Single agent, N=8 votes63.39 ± 0.6084,499
PRISM76.38~3.93,296

N=1 reproduces the paper's vanilla number (61.81) exactly. At PRISM's call budget (N=4) the baseline is 13.8 points lower while using fewer tokens. Doubling the calls (N=8, about 36% more tokens than PRISM) is still 13.0 points lower. Against PRISM's seed-42 outputs, PRISM is correct where the baseline is not on 40-48 questions (N=4) and 43-45 (N=8), against 7-14 in the other direction; every exact McNemar test gives p < 0.001 (N=1: 46 vs 17, p = 0.0003). On AQuA, the gain is therefore not explained by spending more inference on repeated attempts. We have not shown this on GSM8K, where voting may help more, and we will add that counterpart to the paper.

We hope the above addresses your concern, and these discussions will be included in the revised version. If anything is still unclear, we would be glad to elaborate.

Reviewer_cgLjOpenReview ↗

Thanks for the response.

Thanks for your response. It basically addresses my questions. However, it does not change my overall evaluation, and I maintain my original score.

Dear Reviewer cgLj,

Thank you very much for taking the time to read our rebuttal and for your thoughtful feedback. We are glad to hear that our response addressed your questions.

We would like to ask whether there are any remaining concerns or points that you feel are still unclear or insufficiently addressed. We would be more than happy to provide additional clarification, results, or revisions during the discussion period.

If our responses have resolved your main concerns, we would be sincerely grateful if you could consider updating your evaluation accordingly. Of course, we fully respect your judgment and appreciate your careful review, which has helped us improve the paper.

Thank you again for your time and constructive comments.

Best regards,

Authors of Submission 12333

Reviewer_P3iVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PRISM is a post-training method for an autoregressive designer that selects LLM agent roles and communication edges for a given query. It trains a privileged teacher using another execution-verified graph for the same query, supplied through a small, initially zero-output FiLM branch, and then distils the teacher’s decision distributions into a student on trajectories sampled from the student itself. The student combines this distillation objective with supervised anchoring and retains the original designer architecture at deployment, without requiring privileged inputs or additional multi-agent executions during post-training. Across six benchmarks using Qwen2.5-7B-Instruct, the paper reports improvements over existing topology-design methods and continued SFT, while a matched constant-input teacher control supports an additional contribution from successful sibling graphs.

Critical strengths
  1. The method makes practical use of supervision that the existing pipeline already collects
  2. The paper recognizes that supplying another query’s graph can actively mislead the teacher, making a shuffled-input teacher an imperfect null.
Critical weaknesses
  1. Dataset provenance and train–evaluation separation are not specified sufficiently: MIXCHAIN is described through its graph statistics, but its underlying question sources and original dataset splits are not identified. It contains 1,014 queries and supports all four mathematical evaluations. The reader therefore cannot determine whether those queries are disjoint from the evaluated GSM8K, MultiArith, SVAMP, and AQuA questions, or whether overlap and near-duplicates were checked.

  2. The continued-SFT comparison changes both the training objective and the computation budget: PRISM trains a teacher for 40 epochs and a student for 30 epochs with four rollouts per query and teacher rescoring; the SFT control receives 30 supervised epochs. The authors acknowledge this explicitly, which is good, but matching student epochs and learning rate does not isolate the value of the overall post-training recipe.

Questions for authors

See above

2 comments
AuthorsOpenReview ↗

Official response for Reviewer P3iV

It's our great honor to receive Reviewer P3iV's thoughtful comments. We address the concerns below.


[W1] Dataset provenance

Source and selection. MIXCHAIN is built from horseee/MixChain-Z-PRM12K, the math split of PRM800K (12,000 rows, single released train split, so there is no original test split to inherit). We took the first 1,504 records in released order (the remaining 10,494 were never used), split them 40/60 into cold-start and fine-tune sets (601/903), executed the blueprints, and kept every query solved by at least one blueprint. This yields 1,014 queries and 2,947 execution-verified graphs (Appendix C). Inclusion therefore depends on solvability, so the corpus leans toward problems the 7B backbone can solve. The split is by query: 811 train / 203 validation (2,359 / 588 graphs, Table 9). Validation is used only to select the teacher checkpoint and is not drawn from any evaluation set. HumanEval is evaluation-only (Section 5.1). We will add the source and sampling procedure to Section 5.1 and Appendix C.

Overlap audit. GSM8K, MultiArith, SVAMP and AQuA were built independently of the MATH/PRM800K problems underlying MixChain, so MIXCHAIN reaches them by transfer. To avoid relying on this assumption, we compared the 1,014 training queries (and, with identical results, all 11,998 usable MixChain questions) against the four evaluation sets using three checks: exact match after normalisation, shared word 13-grams, and near-duplicate detection (word-3-shingle Jaccard ≥ 0.8). All checks returned zero matches:

Eval setnexact13-gramnear-duplicate
GSM8K1,319000
MultiArith600000
SVAMP1,000000
AQuA254000

Only five shorter 8-gram matches remain (2 on GSM8K, 3 on AQuA). We manually read all five: each is a generic shared phrase inside an otherwise different problem (e.g., "the greatest integer less than or equal to"; "in how many ways can the letters of", with different words and constraints).

Conclusion: the training data does not overlap with any evaluation set (no exact, 13-gram or near-duplicate matches). The audit script and table will be added to the appendix.


[W2] The SFT comparison changes the objective and the computation

We agree that matching epochs and learning rate does not isolate the full recipe; Section 5.1 itself says the settings match the student schedule "but not total training computation." We add:

1. Exact SFT protocol. The Table 1 SFT control was early-stopped on validation NLL (patience 5): 14 epochs, deploying epoch 9 (epochs 6 and 8 for seeds 43 and 44). We will correct the "30 epochs" wording in Section 5.1.

2. Equal-budget SFT. We add final-epoch SFT with 30 epochs (SFT-30, PRISM's student stage) and 70 epochs (SFT-70, teacher + student, 40 + 30), seeds 42-44. Validation NLL worsens with longer SFT (seed 44: 3.07 at epoch 8, 5.61 at epoch 70). Mean accuracy:

GSM8KMultiArithSVAMPAQuAAvg
SFT, early-stopped86.1898.8392.7362.7385.12
SFT-3085.0498.7892.9767.3286.03
SFT-7085.1498.4492.6068.5086.17
PRISM87.7798.8993.4776.5189.16

PRISM's average is 4.0, 3.1 and 3.0 points above early-stopped SFT, SFT-30 and SFT-70. Against SFT-70: +2.6 (GSM8K), +8.0 (AQuA), +0.9 (SVAMP), +0.5 (MultiArith, saturated; Appendix D). AQuA is high-variance (SD 4.5 for SFT-30), so we read it together with the McNemar tests in Table 4. Going from 30 to 70 epochs adds only 0.1 to the SFT average.

3. Cost. On one A5000, SFT takes 2.1 min (30 epochs) and 4.9 min (70); PRISM takes about 10.6 min (3.2 teacher + 7.4 student), roughly twice SFT-70 but small next to the corpus construction both methods share. At inference PRISM makes 3.89 / 3.92 LLM calls per question (GSM8K / AQuA) versus 3.67-3.86 for SFT. Tokens per question: 4,531 vs 3,808-4,188 on GSM8K (+8% to +19%, from slightly larger crews) and 3,296 vs 3,174-3,334 on AQuA (-1% to +4%). PRISM is therefore costlier on GSM8K, and we will report this in the paper.

4. Controls (Section 6.2, Tables 3-4). A constant-input teacher with identical epochs, rollouts, rescoring and loss mixture is beaten by PRISM on all six benchmarks (+1.66% on average, sign test p = 0.031). On AQuA's difficulty-matched control, PRISM wins both where SFT emitted its modal crew (+18.7%, p = 0.0001) and elsewhere (+11.0%, p = 0.003).

We hope this clarifies your concern, and we will reflect these discussions in the revision. We remain happy to address any further questions you may have.

Thank you for your response.

I remain concerned about the empirical evaluation. Given the complexity of the proposed method, fair and comprehensive comparisons with the baselines are particularly important. I do not understand why SFT performance degrades as the compute budget increases, whereas PRISM, despite learning from soft labels, does not exhibit the same degradation. Furthermore, two benchmarks are omitted from the additional experiments, and the gains on the remaining benchmarks are smaller (such as only ~1.5 point improvement on gsm8k). Consequently, I remain unconvinced by the experimental results and uncertain about the usefulness of the proposed method.

All in all, I am sorry that I should keep my point as weak reject.

Comments on the paper

1 comment

Dear Reviewers,

Thank you again for your constructive feedback. We hope that our responses addressed your concerns. As the rebuttal period is coming to an end, we hope that you could acknowledge our responses and consider adjusting your rating in case your concerns are adequately resolved.

Thank you and best regards,

Authors of Submission 12333

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.