acceptodds
Under review as a conference paper at ICLR 2027

Active Learning for Communication Structure Optimization in LLM-Based Multi-Agent Systems

Abstract

Optimizing the communication structure of large language model based multi-agent systems (LLM-MAS) has been shown to improve downstream performance and reduce token usage. Existing methods typically rely on randomly sampled training tasks. However, tasks may differ substantially in difficulty and domain, and thus they are not equally informative for updating communication structure, making optimization often unstable and highly sensitive to the particular training set. To actively identify the most valuable tasks for communication-structure optimization, we propose an ensemble-based information-theoretic task selection framework. The proposed method estimates task informativeness by how much a candidate task changes the distribution over graph parameters, using ensemble Kalman inversion as an efficient and derivative-free approximation of the corresponding Bayesian update. The resulting estimator is especially suitable for black-box and noisy multi-agent systems. To enhance scalability, we construct a compact candidate pool through embedding-based representative selection and combine the informative selection with surrogate modeling and batch Thompson sampling. We validate the proposed framework across both benign and adversarial settings and multiple task formats. It consistently outperforms random training, demonstrating more effective task selection and greater overall cost efficiency.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
active learning, information gain, LLM-based multi-agent systems, communication-structure optimization, ensemble Kalman inversion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FFpDOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper studies active learning for communication-structure optimization in LLM-based multi-agent systems. The main motivation is that existing graph/communication optimization methods usually rely on randomly sampled training tasks, but different tasks are not equally informative for improving the communication structure. The paper empirically shows that communication-structure optimization is sensitive to the selected training tasks, and that simply increasing the number of randomly sampled tasks does not consistently improve downstream performance.

Critical strengths
  1. A key strength is that the paper identifies a practically meaningful weakness in current communication-structure optimization pipelines: the final optimized graph can be highly sensitive to which training tasks are used.

  2. The paper evaluates the method across several task formats, including QA, code generation, tool use, and embodied interaction, and also examines adversarial settings.

  3. The paper does not focus only on mean performance, but also reports run-to-run variability, lower-tail metrics, and token cost.

Critical weaknesses
  1. The main empirical concern is that many of the reported gains over random selection are fairly modest, while the statistical evidence is mixed. In the main benchmarks, improvements in mean accuracy are often small, especially in “benign” settings.

  2. The proposed framework introduces substantial additional machinery, e.g., representative selection, EKI-based utility estimation, surrogate modeling, and batch selection. But the practical advantage over a simpler random-selection baseline is not always strong enough to justify this complexity.

  3. The paper argues for an information-theoretic EKI-based criterion, but it remains unclear whether the reported gains truly come from the specific Bayesian/EKI formulation, or more simply from selecting a more diverse and filtered subset of tasks.

  4. The paper does not convincingly show that the EKI-based utility is substantially more effective than simpler uncertainty-, diversity-, or heuristic-based strategies under matched computational budgets across the full benchmark suite.

  5. Related work is quite insufficient and outdated.

Questions for authors
  1. The paper reports improvements in mean performance, stability, and lower-tail metrics, but several bootstrap confidence intervals include zero. Could the authors clarify which main claims they consider statistically supported and provide significance analysis for all headline comparisons, not only selected cases in the appendix?

  2. How much of the observed gain comes specifically from the EKI-based informativeness criterion, as opposed to the representative pre-selection stage or other pipeline components? Can the authors provide stronger controlled comparisons against simpler uncertainty/diversity-based selectors under matched total token budgets?

  3. The paper emphasizes cost efficiency, but active selection itself introduces nontrivial overhead. Can the authors clarify under what practical budget regimes the proposed method is consistently preferable to simply training on more randomly sampled tasks?

If the authors can address my concerns well, I will be willing to increase the score.

4 comments
AuthorsOpenReview ↗

Response to Reviewer FFpD (1/2): Weaknesses 1-2 and Question 1

We thank the reviewer for the detailed comments and for being open to raising the score. We will revise the paper accordingly.


Weakness 1 and Question 1: size of the gains and statistical support

Q1: Could the authors clarify which main claims they consider statistically supported and provide significance analysis for all headline comparisons, not only selected cases in the appendix?

Claims and support:

  • Mean improvement: supported by the mean on GSM8K under attack and the mean Success@5 on InterCode-SQL.
  • Lower-tail improvement: supported by the Worst-25% mean on MMLU under attack, and by Q1 and the Worst-25% mean on GSM8K under attack.

The table below gives the analysis for all headline comparisons of active against random selection. The lower-tail tests on InterCode-SQL and the intervals on HumanEval are new. ALFWorld has only four to five runs per method. The smallest attainable p-value of a paired permutation test is then 0.0625, which cannot be significant at the 0.05 level, so we do not test significance on ALFWorld. We added this table to Appendix D.3 (Table 8) and stated there which claims are statistically supported (lines 1689–1692). Section 5.2 refers to these tests (line 389).

BenchmarkTestMeanQ1Worst-25% mean
MMLU, benignBootstrap 95% CIincludes 0includes 0includes 0
GSM8K, benignBootstrap 95% CIincludes 0includes 0includes 0
MMLU, attackBootstrap 95% CIincludes 0includes 0excludes 0
GSM8K, attackBootstrap 95% CIexcludes 0excludes 0excludes 0
HumanEvalBootstrap 95% CIincludes 0includes 0includes 0
InterCode-SQL, Success@5Paired permutationp = 0.037p = 0.172p = 0.156
InterCode-SQL, AUC@5Paired permutationp = 0.066p = 0.133p = 0.156

W1: Many of the reported gains over random selection are fairly modest ...

Why some gains are smaller:

For some benchmarks (typically the easier ones), there is not much room for improvement. On these benchmarks, graph optimization with randomly selected tasks improves performance by only about 1 pp, similar to Zhang et al. (2025b). Active selection can at most reach what graph optimization can achieve. Its gain over random selection comes from avoiding the poor task sets that lead to failure cases. This lifts the lower tail and, with it, the mean. So the mean gains on these benchmarks are expected to be small.

However, on the new benchmarks that we add and the original work (Zhang et al., 2025b) does not use, i.e., InterCode-SQL and ALFWorld, the improvements are relatively large (typically 3–8 pp).

We added this explanation as Appendix D.3.1.


Weakness 2: complexity

W2: The proposed framework introduces substantial additional machinery ...

The key feature of our method is to use EKI to approximate information gain for task selection. All other components are used to save computational budget, as stated at the beginning of Section 4.5. Readers can remove the other components: EKI-based information gain alone can still select good tasks (Appendix D.6), only at the cost of more selection tokens. We added this point to Section 4.5 (lines 331–332).

AuthorsOpenReview ↗

Response to Reviewer FFpD (2/2): Weaknesses 3-5 and Question 2-3


Weakness 3 and Question 2 (first part): ablation against diversity-based criteria

W3: It remains unclear whether the reported gains truly come from the specific Bayesian/EKI formulation, or more simply from selecting a more diverse and filtered subset of tasks.

Q2 (first part): How much of the observed gain comes specifically from the EKI-based informativeness criterion, as opposed to the representative pre-selection stage or other pipeline components?

We have an ablation study in Appendix D.6 that compares diversity only, informativeness only, and their combination. It shows that informativeness-only selection improves more than diversity-only selection. Nevertheless, our aim is to show that both are necessary for an efficient and effective method, not to compare them.


Weakness 4: informativeness vs. uncertainty-, diversity-, or other heuristic-based criteria

W4: The paper does not convincingly show that the EKI-based utility is substantially more effective than simpler uncertainty-, diversity-, or heuristic-based strategies under matched computational budgets across the full benchmark suite.

We do not argue that informativeness is better than other non-informativeness criteria. We focus on how to efficiently estimate information gain, so we compare different ways to estimate it (Section 5.4).

We simply chose a two-stage strategy, diversity first and then informativeness, as is common practice. Other combinations may perform better. We leave the search for the best combination to future work.


Question 2 (second part): cost-matched comparison

Q2 (second part): Can the authors provide stronger controlled comparisons against simpler uncertainty/diversity-based selectors under matched total token budgets?

The reviewer asks for a comparison against uncertainty- and diversity-based selectors under matched total token budgets.

Note that the diversity-based selector costs no tokens (it uses only the tasks themselves and does not run LLM rollouts), so we cannot conduct a matched-token comparison against the informativeness-based selector.

For the uncertainty-based selector, the reason is the same as in our reply to Weakness 4: this work does not claim that informativeness is better than uncertainty. Its focus is how to efficiently estimate information gain for task selection.


Question 3: cost efficiency

Q3: Can the authors clarify under what practical budget regimes the proposed method is consistently preferable to simply training on more randomly sampled tasks?

Table 1 (Section 5.2) shows that, under matched total cost, active learning achieves better downstream performance than training on randomly selected tasks.

Here, the total cost of active learning (additional selection cost + training cost) equals the total cost of random selection (training cost).

We have made the Table 1 easiler to understand.

Moreover, random selection does not reach active learning at any training budget. In Fig. 1, we test random selection from 0.05M to 3.54M training tokens, while the total cost of active learning is fixed at about 0.9M tokens. Random selection's mean accuracy first increases and then decreases. Even at its peak, it stays below the mean accuracy of active learning.

We interpret the question as asking for the break-even point, where both methods reach the same performance at the same total cost. This point answers when to use active learning, while our work answers how to perform active learning. Finding the break-even point is an interesting problem, and we leave it for future work. We now state it as a limitation in Section 6 (lines 507–509) and leave it for future work.

Weakness 5: related work

W5: Related work is quite insufficient and outdated.

We expanded Section 3 with recent work on data selection for LLMs (Xia et al., 2024; Zhang et al., 2025c; Lin et al., 2025; Chen et al., 2025) and recent information-theoretic criteria (Guilhoto et al., 2026; Durmazkeser et al., 2026; Huang et al., 2026) (lines 126–129). We would also welcome any specific references the reviewer has in mind.

For communication-structure optimization, the introduction cites the latest methods published before May 2026. We also checked for newer work and added Xu et al. (2026) in Section 3 (line 121).


We hope these responses address the reviewer’s concerns. If so, we would be grateful if the reviewer would consider raising the score.

Thanks for the author's response. Here is my opinion:

Q1 actually confirmed rather than disproved W1. It indicates that the significant positive results were mainly confined to the "attack" scenario and the "InterCode-SQL" new benchmark, while there was no significant result in the "benign" scenario.

W5 has responded appropriately.

W2 The author states that EKI is the core component, and the other components are merely for budget savings and can be removed (Appendix D.6), yet the selection of tasks can still be done with EKI, only at the cost of more tokens. This is a reasonable narrowing defense, but it still fails to provide a direct argument regarding the cost-effectiveness of this complex approach compared to "simply using more random tasks".

Q3 the author bypassed the core of my question - "In what budget range is this method more stable and effective?" - instead of providing an answer.

Overall, some responses (Q1, W5) were honest and thorough, while Q3 and W2 made substantive progress; however, regarding your core methodological doubts (W3, W4, and Q2), the replies were evasive by "redefining the scope of the claim" rather than providing answers. Therefore, from the perspective of "whether they accurately answered your questions": The claims were somewhat comprehensively and honestly narrowed, but the two most critical empirical gaps were not addressed. If based only on this reply, I don't think it is sufficient to raise the score further from "Weak rejection". My current highest rating is "Weak rejection".

AuthorsOpenReview ↗

Response to Follow-up Comments

We thank the reviewer for raising your score and for the follow-up comments. Our responses are below.


Q1 / W1: Benign scenario

Reviewer: Q1 actually confirmed rather than disproved W1. It indicates that the significant positive results were mainly confined to the "attack" scenario and the "InterCode-SQL" new benchmark, while there was no significant result in the "benign" scenario.

Response. We agree with this observation. We believe the main reason is that the benign scenario leaves little room for improvement, as explained in Appendix D.3.1.

In the revision, we propose to make the following changes:

  • We will narrow the main claim of the paper and focus it on the attack scenario.
  • We will move the benign results to the appendix.

Do they address your concern?


W5

Reviewer: W5 has responded appropriately.

Response. Thank you for confirming.


W2: Cost-effectiveness compared with more random tasks

Reviewer: The author states that EKI is the core component, and the other components are merely for budget savings and can be removed (Appendix D.6), yet the selection of tasks can still be done with EKI, only at the cost of more tokens. This is a reasonable narrowing defense, but it still fails to provide a direct argument regarding the cost-effectiveness of this complex approach compared to "simply using more random tasks".

Response. We understand the question as: under the same total cost, does our approach perform better than simply using more random tasks?

Yes, Fig. 1 and Table 1 answer this question. Under matched total token cost, our approach outperforms random task selection on every metric for both benchmarks:

BenchmarkMetricOursRandomΔ
MMLUMean78.32%77.35%+0.97 pp
Q176.47%75.82%+0.65 pp
Worst-25%76.25%73.85%+2.40 pp
GSM8KMean93.83%92.64%+1.19 pp
Q193.15%92.11%+1.04 pp
Worst-25%92.93%91.73%+1.20 pp

Since the token cost is the same, we believe this is a direct evidence that our approach is more cost-effective than using more random tasks.


Q3: Across budget ranges

Reviewer: the author bypassed the core of my question - "In what budget range is this method more stable and effective?" - instead of providing an answer.

Response. We now understand the question as: when the total budget changes, does our approach still outperform cost-matched random selection? Yes.

We run the cost-matched test on more budget levels. We did not have time to test a wider and denser range of budgets, since we received this clarification less than one day ago. We were able to add a few budget levels, which took several hours to run. We report results on MMLU (benign), since the reviewer's concern is mainly about the improvement in this setting:

Total cost (M tokens)Ours vs. RandomRuns
~1.281.96% vs 81.37% (↑)10 vs 6
~2.281.45% vs 80.70% (↑)10 vs 6
~4.382.29% vs 81.48% (↑)10 vs 3

At each matched cost, our approach achieves a higher mean accuracy than random selection, by 0.59 pp, 0.75 pp, and 0.81 pp at about 1.2M, 2.2M, and 4.3M tokens, respectively. We don't have time to revise the paper. We will include this results later.


Comparison with cost-matched uncertainty-based sampling. In case the question also concerns how our information-gain criterion compares with uncertainty sampling, we ran a new comparison against cost-matched uncertainty-based sampling at three cost levels on MMLU benign (each for 10 runs).

Cost (M tokens)MethodMeanQ1Worst-25%
level 1 (~1.2)Ours81.96% (↑)81.05% (↑)80.78% (↑)
Uncertainty80.83%79.74%79.74%
level 2 (~2.2)Ours81.45% (X)80.39% (↑)78.98% (X)
Uncertainty81.70%79.74%79.30%
level 3 (~4.3)Ours82.29% (↑)81.86% (↑)81.18% (↑)
Uncertainty80.39%79.08%78.65%

At cost levels 1 and 3, our approach outperforms uncertainty sampling by at least 1 pp on all three metrics. At cost level 2, uncertainty sampling is slightly better on Mean and Worst-25% (by 0.25 and 0.32 pp), while our approach is better on Q1 (by 0.65 pp).

This comparison is not the focus of our work, and we do not claim that either criterion is better in general.


We hope our responses address your two main concerns:

  • the modest improvement in the benign scenario;
  • the comparison with cost-matched random selection at different budgets.

If so, we would be grateful if you would consider raising your score further.

Reviewer_biuyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

this paper study a relatively under-explored problem. They authors want to figure out the inner connection between task selection and agents system performance and they propose to use active learning framework to solve that. The contributions may include their findings about task selections towards agent structure and their active-learning-based approach.

Critical strengths

the most positive one to support my potential acceptance is their perspective is interesting, and seems to be novel, compared with current too many agent structure papers.

Critical weaknesses

the most critical weakness of the paper that lead to my rejection is that: their findings and their approaches, seem to be not new out of the scope of LLM-based multi-agent system.

a second concern is that some motivations are missing, for example, in section 4.2 "We approximate the Bayesian update with ensemble Kalman inversion", and why?

Questions for authors

I may miss something, it would be greatly appreciated if the authors can clearly tell me what exactly theory contribution (if they have)? what exactly non-theory contribution? what is the unique thing if I move their framework from LLM-based agent system to other scenario?

4 comments
AuthorsOpenReview ↗

Response to weakness 1 and question

We thank the reviewer for finding our perspective interesting and novel. As requested, we first list our contributions and then state what is unique beyond LLM-MAS.


Contributions

  1. We identify the problem, give evidence for it, and show that it can be solved. We first show evidence that for graph optmization of LLM-MAS, downstream accuracy varies widely across random task sets, and a larger random budget does not remove the variation (Fig. 1). This may sound like common sense, but to our knowledge it has not been reported with evidence before.
  2. We show which methods work for this problem and which do not. Under the same selection budget, several common active learning criteria perform close to random selection. The EKI-based information gain and the Fisher-determinant coreset perform best (Section 5.4, Table 14). We also explain when to use each of the two: EKI needs no gradients and less memory, and Fisher-determinant selection can be better when gradients can be estimated accurately. Section 5.4 now summarizes this discussion (lines 461–463), and Appendix D.5.7 gives the details.
  3. We add to the understanding of EKI-based information gain in general, not only on LLM-MAS.
    • It selects learnable tasks. A task on which all sampled graphs score the same receives zero gain, whether they all solve it or all fail. A task on which different graphs score differently is one that can change the graph. This follows from Eq. (8) (example: Task 11587 in Table 16). The introduction also states it (lines 073–074).
    • It works with a small ensemble. Task selection remains effective when the ensemble is much smaller than the parameter dimension (6 particles for 36 parameters, 10 for 100, and 15 for 400; Appendix D.1.1). To our knowledge, this is the first test and the first evidence of EKI-based data selection with such a small ensemble. Prior work on EKI-based information gain used 40 particles for 37 parameters (Yang et al., 2026a). We added Section 5.5 to report this result in the main text, and list it as our third contribution (lines 089–091).
    • It extends to batch selection. We provide the multi-observation form for selecting a batch of tasks (Section 4.4). Prior work on EKI-based information gain considers one observation at a time, namely the position of a single sensor.

What is unique beyond LLM-MAS. We answer this question in two parts.

  1. What the general method needed for this application.
    • We drop the outer expectation. EKI-based information gain comes from Bayesian experimental design, where the expected information gain is used for scientific computing problems governed by PDEs. In our application, the task–answer pairs are already available. The information gain itself is then sufficient, and the outer expectation is not needed (Section 4.1, lines 221–228). This also distinguishes our method from BALD, which also comes from Bayesian experimental design.
    • We set the “observation” in EKI to the desired score (Section 4.2, line 270). In PDE problems, the observation is a measurement of a true state. No such ground truth exists for a communication graph.
  2. What we add to the general method. See the third contribution above.

We hope these responses address the reviewer’s first concerns. If so, we would be grateful if the reviewer would consider raising the score.

AuthorsOpenReview ↗

Response to weakness 2

Why EKI


A full Bayesian update, such as MCMC or SMC, is too expensive for LLM-MAS, so we need a cheap substitute. We moved this statement from the end of Section 4.1 to the start of Section 4.2 (lines 236–238) to make it more visible.

EKI is a good substitute for two reasons. First, it works with a small ensemble, much smaller than the parameter dimension, so each task costs only a few rollouts. Second, it needs no gradients. It only runs the MAS forward, so the MAS can stay a black box. Other approximations, such as the Laplace approximation or variational inference, need gradients, and gradients are available here only as noisy policy-gradient estimates.

thanks for the response. I will maintain my score.

Thanks for letting us know. If any concerns remain, we would be glad to clarify them.

Reviewer_VuD2OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to select training tasks for communication-structure optimization in LLM-based multi-agent systems. The authors observe that randomly choosing training tasks can lead to unstable optimized communication graphs and large variation in downstream performance. The proposed method treats task selection as an active learning problem. It first reduces the candidate task pool using embedding-based representative selection, and then estimates the informativeness of each remaining task through the change it induces in the distribution of communication-graph parameters. Ensemble Kalman inversion (EKI) is used as a derivative-free approximation of the Bayesian update, and the resulting distribution change is measured with KL divergence. A surrogate model is further used to reduce the cost of evaluating task utility. The method is evaluated on MMLU and GSM8K in benign and adversarial settings, and further tested on HumanEval, InterCode-SQL, and ALFWorld. The results show that active task selection generally improves downstream performance and stability compared with random task selection. The paper also compares with several other informativeness-based active learning methods and includes sensitivity, scalability, and ablation studies.

Critical strengths
  1. I found the empirical motivation convincing. The paper shows that communication-structure optimization can change appreciably depending on which tasks are used for training, and that simply adding more randomly selected tasks does not reliably fix the problem. That observation is easy to miss if one focuses only on designing a better graph optimizer. It provides a good reason to study task selection as a separate problem.
  2. The experiments take the cost of selection seriously. The authors compare selection plus training against random training rather than treating the task-scoring stage as free, and they examine benign and adversarial settings as well as transfer to other task formats. I also appreciated the comparisons with EGL, EMC, and Fisher-based criteria. The run-to-run and lower-tail analyses are useful here because instability is part of the paper's initial motivation.
Critical weaknesses
  1. The weakest step, in my view, is the interpretation of the EKI score as information gain. In the implementation described in Appendix B, the optimized graph parameter has 36 dimensions but the ensemble has only six particles. The KL calculation therefore uses coordinate-wise Gaussian means and variances rather than a full covariance estimate. I understand why the authors made this choice; a sample full covariance would be rank-deficient. The issue is not numerical feasibility, but whether the resulting ranking measures anything close to the proposed Bayesian quantity. The current sensitivity experiments address ranking stability to some extent, not the fidelity of the posterior or information-gain approximation.
  2. The evidence for selecting informative tasks is stronger than the evidence that EKI is the best way to select them. In the MMLU attack comparison (Appendix D, Table 12), Fisher-determinant selection has a mean of 78.59% versus 78.32% for EKI, with nontrivial variation for both. The authors appropriately qualify the Fisher comparison in the appendix. I would carry that qualification into the main discussion and make the contribution less dependent on the particular estimator winning every comparison.
  3. One terminology point: these are candidate task-answer pairs with reference answers already available. The method chooses which examples to use for graph optimization, rather than deciding which unlabeled examples to annotate. 'Active learning' is not necessarily wrong as a broad framing, but readers from that literature may expect label-acquisition costs to enter the objective. A short clarification would avoid confusion.
Questions for authors
  1. How stable are the selected task rankings when the ensemble is larger? Even on a reduced graph-parameter space, a comparison between six particles and a better-conditioned covariance estimate would help me judge the EKI/KL approximation.
  2. Are the differences between EKI and the stronger Fisher-based baseline meaningful given the reported variation? I would welcome a paired comparison or a more explicit discussion of when each selection rule works better.
  3. Can you clarify what information is assumed to be available for every candidate task before selection? In particular, are reference answers always given, and is there any label-acquisition cost in the setting considered?
3 comments
AuthorsOpenReview ↗

Response to Weakness 1 and Question 1

Overall: We sincerely thank the reviewer for the careful and constructive review. The suggestions gave us clear directions for answering these questions and improving the paper. We hope the responses below address the reviewer’s concerns. If so, we would be grateful if the reviewer would consider raising the score.


W1/Q1:

We understand the concern as having two parts: (1) whether EKI accurately approximates the Bayesian posterior and information gain, and (2) whether the diagonal Gaussian KL approximation with a small ensemble produces reliable task rankings. We first address the second concern with additional experiments.

We reduce the parameter space by randomly selecting 4 graph parameters to consider information gain and freezing the others. We compare two KL estimators: (a) the original diagonal KL, and (b) the full-covariance KL, with a small ridge term for J=3J = 3. We first evaluate the stability of task selection across ensemble sizes, using J=20J = 20 as the reference. We then compare the Top-10 overlap between the two estimators at the same ensemble size.

JStability: Diag.Stability: FullDiag. vs. Full
30.722 ± 0.1300.733 ± 0.0710.833 ± 0.112
60.722 ± 0.0830.711 ± 0.1050.811 ± 0.093
120.744 ± 0.1130.667 ± 0.0870.911 ± 0.078
20 (as reference)1.0001.0000.911 ± 0.078
20 (vs another run)0.700 ± 0.0710.733 ± 0.050–

Values are mean ± standard deviation over nine runs (3 task pools × 3 seeds, one parameter set per task pool). Stability is the Top-10 overlap with the J=20J = 20 selection of the same estimator. The last row compares two J=20J = 20 runs with different seeds on the same task pool. We added these results and the full experimental details to Appendix D.4.2 (Table 13). Section 5.5 summarizes them (lines 472–477, Fig. 7).

The diagonal approximation has little impact on task selection. At J=12J = 12 and 2020, where the full covariance is well-conditioned, the diagonal and full KL share 9.1 of 10 selected tasks on average.

The ensemble size does not noticeably affect task selection. For J=3J = 3 to 1212, the selection shares about 7 of 10 tasks with J=20J = 20, the same as two J=20J = 20 runs with different seeds (last row). This overlap is well above the 2 of 10 expected from random selection.

Regarding the first concern, we agree that EKI does not generally recover the exact Bayesian posterior. It is exact only for linear-Gaussian models. We regard the EKI-based KL as a computationally efficient proxy for task selection, rather than an accurate estimator of the true Bayesian information gain. We empirically show that this proxy is effective for task selection. We clarified this distinction in Section 4.3 (lines 287–289) and stated it as a limitation in Section 6 (lines 504–507).

AuthorsOpenReview ↗

Response to Weakness 2 and Question 2

We thank the reviewer for this suggestion. In short, EKI and Fisher-determinant selection perform comparably in this experiment, and each rule is better suited to a different setting.

Statistical comparison. The existing results were not paired across methods by random seed (Appendix D.4), so a paired statistical test is not applicable to this comparison. We therefore computed a bootstrap 95% confidence interval for the difference in mean accuracy, EKI minus Fisher-determinant selection, using the same procedure as in Appendix D.3.3. The interval is [−2.17, 1.53] percentage points and includes zero. The two methods are not statistically distinguishable in this experiment.

When each rule works better. Fisher-determinant selection can work better when gradients can be estimated accurately, for example, when the forward model is differentiable. It can also work better when candidate tasks provide overlapping information, since the coreset avoids redundant tasks. EKI can work better when gradients are unavailable or noisy, or when the Fisher matrix is costly to store (O(N4)O(N^4) memory for an NN-agent graph, versus O(JN2)O(JN^2) for EKI).

Revision. Section 5.4 now states that EKI and Fisher-determinant selection perform best among the compared criteria and are not statistically distinguishable in this experiment (lines 460–461). Appendix D.5.5 states that we do not claim EKI generally outperforms Fisher-based selection (lines 2042–2044). In the contribution list, we now present the main contribution as selecting informative tasks for graph optimization, with EKI as one way to measure informativeness (lines 081–088). Appendix D.5.7 summarizes when each rule works better, and Section 5.4 refers to it (lines 461–463).

AuthorsOpenReview ↗

Response to Weakness 3 and Question 3

We thank the reviewer for pointing out this possible confusion. In our setting, every candidate task comes with its reference answer, and there is no label-acquisition cost. The method selects which task–answer pairs to use for graph optimization. We aim to reduce the cost of forward model evaluations (LLM rollouts) in graph optimization, not the cost of annotation.

This setting is stated in Section 4.1 and in the limitations. With the reference answer available, we compute the realized information gain for each candidate task, and no expectation over possible answers is needed. For unlabeled tasks, an outer expectation over possible answers would recover the standard expected information gain (or BALD), and label-acquisition cost would also need to be considered. We leave this setting to future work on task generation.

We agree that this point should be stated earlier and more clearly. We now state it at the start of Section 3 (lines 118–122), before the method is introduced.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response and Summary of Revisions

We thank all reviewers and the Area Chair for their constructive feedback. We have responded to each review in detail and revised the manuscript accordingly. Key changes in the main text are highlighted in blue. Below we first summarize the reviewers' concerns and our responses across three themes: method and scope clarifications, an additional experiment, and additional analyses.

1. Method and scope clarifications.

  • Problem setting: Stated at the start of Section 3 that every candidate task comes with its reference answer. The method therefore uses the realized information gain, and it reduces training cost rather than labeling cost (VuD2 W3/Q3; biuy W1).
  • EKI-based utility: Added Section 4.3 on its exactness, its preference for learnable tasks, and its role in training (VuD2 W1/Q1; biuy W1).
  • Why EKI: Moved the reason for using EKI instead of a full Bayesian update to the start of Section 4.2 (biuy W2).
  • Contributions: Rewrote the contribution list in Section 1. (VuD2 W2/Q2; biuy W1).
  • Small ensembles: Added Section 5.5, which brings the existing results on larger systems (Appendix D.1.1) and on ensemble size (Appendix D.4.1) into the main text (biuy W1; VuD2 W1/Q1).
  • Pipeline complexity: Stated in Section 4.5 that representative pre-selection and surrogate modeling only reduce cost and can be removed (FFpD W2).
  • Scope of comparison: Clarified that we compare ways to estimate information gain, not informativeness against other criteria, and pointed to the existing diversity-versus-informativeness ablation in Appendix D.6 (FFpD W3, W4, Q2).
  • Cost efficiency: Made Table 1 clearer, and stated the break-even point between active and random selection as a limitation in Section 6 (FFpD Q3).
  • Other limitations: Stated in Section 6 that the EKI-based KL is a proxy for the information gain, and that settings requiring label acquisition or task generation are left for future work (VuD2 W1, W3).
  • Related work: Added recent work to Section 3 (FFpD W5).

2. Additional experiment.

  • KL approximation: Compared the diagonal KL used in our method with the full-covariance KL in a reduced setting with 4 graph parameters. The two estimators share 8 to 9 of the 10 selected tasks. The same test also shows that selection with J=3J = 3 to 1212 agrees with J=20J = 20 about as well as two J=20J = 20 runs with different seeds (Appendix D.4.2, Table 13; Section 5.5, Fig. 7) (VuD2 W1/Q1).

3. Additional analyses.

  • Statistical significance: Added tests for all headline comparisons in Appendix D.3 (Table 8), and stated which claims are statistically supported (FFpD W1/Q1).
  • Size of the gains: Explained in Appendix D.3.1 why the gains are smaller on easier benchmarks (FFpD W1).
  • EKI vs. Fisher-determinant selection: Added a bootstrap confidence interval showing that the two are not statistically distinguishable, and discussed when each method works better (Section 5.4; Appendices D.5.5 and D.5.7) (VuD2 W2/Q2).

In addition, the table below organizes our manuscript revisions by reviewer concern and lists their locations.

Reviewer concernsRevision / clarificationLocation
VuD2 W1, Q1Added diagonal vs. full KL comparison; summarized existing ensemble-size results in the main text; stated that the EKI-based KL is a low-cost surrogate; added limitationSection 4.3; Section 5.5; Fig. 7; Appendix D.4.2; Table 13; Section 6
VuD2 W2, Q2Added bootstrap comparison with Fisher-determinant selection; revised claims; discussed when each method works betterSection 1; Section 5.4; Appendix D.5.5; Appendix D.5.7
VuD2 W3, Q3Clarified that task–answer pairs are given and the realized information gain is used; added limitationSection 3; Section 4.1; Section 6
biuy W1, Q1Rewrote contribution list; added remark on learnable tasks; moved small-ensemble results to the main textSection 1; Section 4.3; Section 5.5
biuy W2Clarified why EKI is usedSection 4.2
FFpD W1, Q1Added statistical tests for all headline comparisons; explained smaller gains on easier benchmarksSection 5.2; Appendix D.3; Table 8; Appendix D.3.1
FFpD W2Stated that pre-selection and surrogate modeling are optional cost-saving componentsSection 4.5
FFpD W3, Q2 (first part)Pointed to the existing ablation of diversity-only and informativeness-only selectionAppendix D.6
FFpD W4, Q2 (second part)Clarified that the paper compares ways to estimate information gainResponse to W4; Section 5.4
FFpD Q3Made Table 1 clearer; added the break-even point as a limitationTable 1; Section 6
FFpD W5Added recent related workSection 3

Detailed responses to each concern are provided under the corresponding review. We sincerely appreciate the reviewers' time and effort in helping us improve the paper, and we hope these revisions address their concerns.

Best regards,

Authors

AuthorsOpenReview ↗

Update on Follow-up Discussion with Reviewer FFpD

Dear Action Editor,

We would like to briefly update you on our response to Reviewer FFpD's follow-up comment. We have replied with new experiments and planned revisions.


Experiments and the questions they answer

  • (Existing) Cost-effectiveness compared with more random tasks (W2). Our existing results (Fig. 1 and Table 1) answer this question. Under (current) matched total token cost, our approach outperforms random task selection on every metric for both MMLU and GSM8K.
  • (New) Performance across budgets (Q3). The reviewer asked when the total budget changes, whether our approach still outperforms cost-matched random selection. We ran new cost-matched experiments at three budget levels (about 1.2M, 2.2M, and 4.3M tokens) on MMLU (benign). At each new budget, our approach achieves a higher mean accuracy than random selection.
  • (New) Comparison with uncertainty-based sampling. We also ran a new comparison with cost-matched uncertainty-based sampling at the same three budget levels. Our information-gain criterion performs better in 7 out of 9 comparisons. Even so, we do not claim that either criterion is better in general. We include this comparison only in response to the reviewer's interest.

Planned revisions

To address the reviewer's concern about the modest improvement in the benign scenario, we will make two changes to the paper:

  • We will narrow the main claim of the paper and focus it on the attack scenario.
  • We will move the benign results to the appendix, so that the main text matches the narrowed claim.

We did not have time to revise the paper during the discussion period. We commit to including these changes and the new results in the revised version.


Thank you for your time and for handling our paper.

Best,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.