acceptodds
Under review as a conference paper at ICLR 2027

Organizing Collaboration Around Contributions: Agentic Intellectual Property for Long-Horizon Multi-Agent Systems

Abstract

Long-horizon tool use requires more than generating and combining candidate responses: agents must identify which contributions add grounded information to subsequent decisions. We introduce Agentic Intellectual Property (AIP), a framework that organizes multi-agent collaboration around attributable contributions rather than primarily around agent identities, specialized roles, or communication topology. At each environment state, agents contribute Action, Evidence, or Synthesis, enabling collaboration through executable proposals, supporting or opposing reasons, and new decision-relevant conclusions. Independent peer reviews assess contribution novelty, usefulness, and validity, assign credit, and govern admission to a shared ledger. An independent Final Judge uses the current state and retained contributions to generate the sole action committed to the environment. This design encourages useful exploration while seeking to limit the propagation of redundant or unsupported content, using the same backbone across all roles and no counterfactual environment branches. Evaluations on BFCL v4 multi-turn and ToolHop show that AIP achieves the highest overall accuracy among all evaluated single-agent and multi-agent baselines with both Qwen3-4B-Instruct and Llama-3.1-8B-Instruct. AIP reaches 17.5% and 40.1% accuracy with Qwen, and 14.3% and 11.0% with Llama, on the two benchmarks, respectively, while recording fewer average tool errors per task than multiple competing architectures. These results support contribution representation, evaluation, and selective propagation as a complementary design axis for improving long-horizon multi-agent collaboration.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
multi-agent systems, language model agents, multi-agent collaboration, long-horizon tool use, function calling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aK2qOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Agentic Intellectual Property (AIP), a multi-agent framework that aims to improve long-horizon tool use through contribution recognition and peer review. Agents generate Action, Evidence, or Synthesis contributions, which undergo peer review before being selectively incorporated into a shared ledger to inform a Final Judge. Experiments on BFCL and ToolHop demonstrate improved accuracy over existing single-agent and multi-agent baselines.

Critical strengths
  1. Practical and relevant problem. The paper addresses an important challenge in long-horizon multi-agent tool use by exploring contribution recognition, peer review, and selective information sharing.

  2. Illustrative case studies. The paper provides concrete examples of how agent contributions can help correct errors and inform subsequent tool-use decisions, offering useful insights into the proposed framework's behavior.

Critical weaknesses
  1. Insufficient methodological justification. While AIP is motivated by contribution recognition and attribution, it remains unclear how faithfully peer-assigned credit reflects each agent's actual contribution to the outcome. The paper lacks direct empirical validation of this attribution mechanism. Moreover, despite the peer-review process, the final decision ultimately depends on the Final Judge, which can independently modify or disregard accepted contributions. This raises questions about how much the proposed credit mechanism influences the final decision. More broadly, AIP shares similarities with existing critique-and-aggregation frameworks [1,2,3], and its methodological contribution is not sufficiently justified.

  2. Insufficient empirical analysis. While the authors evaluate AIP on BFCL and ToolHop, the experimental evaluation has several limitations:

    • Marginal improvements, computational budgets, and generalizability. Some improvements over the strongest baselines are relatively small (e.g., 40.1% vs. 39.7% on ToolHop), yet the main results lack confidence intervals or significance tests to establish their robustness. Moreover, it is unclear whether the comparisons are fair in terms of computational budgets. For example, AIP requires up to 50 model calls per decision state, but no computation-matched comparisons are provided to determine whether the improvements persist under comparable inference budgets for different baselines. Finally, given that AIP is proposed as a general multi-agent collaboration framework, the authors do not sufficiently justify why the evaluation is restricted to two tool-use benchmarks.

    • Missing ablation analysis and controlled comparisons. The individual contributions of Action, Evidence, and Synthesis remain unclear, as the Action-only ablation removes both Evidence and Synthesis simultaneously. Moreover, AIP employs seven contributors and a separate Final Judge, while the baselines adopt different agent configurations. More controlled comparisons are needed to isolate the effects of individual components and establish whether the improvements arise from the proposed contribution-centered mechanism rather than differences in agent configurations.

Reference


  1. Liang, Tian, et al. "Encouraging divergent thinking in large language models through multi-agent debate." EMNLP 2024
  2. Wang, Junlin, et al. "Mixture-of-agents enhances large language model capabilities." ICLR 2025
  3. Qian, Chen, et al. "Scaling large language model-based multi-agent collaboration." ICLR 2025
Questions for authors

See weakness

Reviewer_vz9QOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces Agentic Intellectual Property (AIP), a contribution-centered framework for improving collaboration in long-horizon multi-agent systems. The authors argue that unlike existing approaches, AIP focuses on evaluating and selectively propagating individual agent contributions. The main contribution of the framework is a collaboration protocol in which agents generate three types of contributions: action, evidence, and synthesis. At each decision step of the agentic system, the first agent proposes an initial action (referred to as the anchor), followed by six other agents that sequentially generate additional contributions if they have anything useful to add. Each generated contribution undergoes an independent peer review based on five evaluation criteria. Contributions receiving positive credit are admitted to a shared ledger, which an independent Final Judge uses to determine the next action. Only the Judge’s selected action is executed in the environment. The authors evaluate AIP on the BFCL v4 and ToolHop benchmarks showing that their method achieves highest overall accuracy among seven evaluated baselines with improvements in tool-execution errors in several settings.

Critical strengths
  • Applying intellectual-property concepts to multi-agent collaboration is an interesting idea that offers a different perspective on how agent contribution can be analyzed and evaluated.
  • Allowing evidence and new insights (not only actions) to be contributions is a useful design choice, it takes into account that agents can improve a task without proposing a concrete action.
  • Separating contribution evaluation from execution allows AIP to explore alternatives within a single execution trajectory, without requiring repeated system runs or environment resets.
Critical weaknesses
  • The novelty is not sufficiently established. DyLAN [1] already uses peer evaluation and contribution-based scoring, but the paper does not discuss it or compare against it. The paper therefore does not clearly establish what AIP adds beyond existing contribution-evaluation methods.

  • The evaluation does not separate the benefit of AIP’s design from the benefit of additional computation. AIP uses up to 50 model calls per decision state, yet there is no resource matching baseline comparison or systematic cost and latency analysis.

  • The reliability of the performance gains is not sufficiently established. Each task is evaluated with only one trajectory, without any task repetition to examine variability and stability.

[1] Liu, Z., Zhang, Y., Li, P., Liu, Y., & Yang, D. (2023). A dynamic LLM-powered agent network for task-oriented agent collaboration. arXiv preprint arXiv:2310.02170.

Questions for authors
  • How does AIP differ from DyLAN beyond scoring individual contributions and not agents?

  • Could the authors compare AIP with DyLAN under the same inference budget?

  • Do AIP’s accuracy gains persist if baselines can spend the same computational budget on additional attempts to solve the task?

  • What are the average LLM-call counts, input/output token counts, and end-to-end latency per task for AIP and each baseline?

  • Are the gains consistent across repeated runs with different seeds? Please report mean and standard deviation.

  • How many BFCL tasks were evaluated in the main experiments and each ablation? Were all 800 tasks always used, and if not, were task sets identical across methods?

  • How accurately do peer reviews identify useful, grounded contributions? Can you provide an independent assessment of both admitted and rejected contributions?

  • Do six reviewers using the same backbone provide meaningfully different judgments? How does performance change with fewer reviewers or when using different backbones?

  • Why is a unanimous positive vote required for admission? How does this compare with majority voting, specifically in rejecting useful contributions?

Reviewer_CB9pOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper organizes multi-agent collaboration around attributable contributions, with peer review, synthesis, explicit abstention, and a separate judge selecting the executable action. It reports the best accuracy across BFCL and ToolHub with 4B and 8B models.

Critical strengths

The contribution-centered design is distinctive: it changes what information influences subsequent actions and allows agents to abstain. The execution boundary is clear, and the three ablations support the complete pipeline.

Critical weaknesses

The method uses roughly 50 calls per decision versus one for a plain agent and five for self-consistency. Without compute-matched baselines, the gains cannot be separated from additional budget. Several margins are only 0.4–1.2 percentage points, without uncertainty estimates. Accuracy granularity also raises questions about the stated evaluation size, while two exactly-zero Llama baseline scores warrant checking formatting and parsing failures.

Questions for authors

Please provide compute-matched comparisons, confidence intervals, exact evaluation denominators, and a failure breakdown for the zero-scoring baselines.

Comments on the paper

1 comment
Area_Chair_fnXiOpenReview ↗

Gentle Reminder: ICLR Rebuttal Discussion Deadline – November 18

Dear Reviewers,

Thank you very much for your time, thoughtful reviews, and valuable contributions to the evaluation process. Thank you to the reviewers who have already interacted and provided additional feedback to the authors!

The authors have carefully addressed your comments and concerns, and the discussion period will conclude on November 18. We would greatly appreciate it if you could take a few moments to review their responses and results, engage in the discussion, and indicate whether their clarifications or additional evidence affect your assessment.

Your continued engagement is essential to ensuring a rigorous, constructive, and fair evaluation process.

Thank you again for your time, expertise, and dedication!

Best regards, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.