acceptodds
Under review as a conference paper at ICLR 2027

AgentPedia: A Collaborative and Compute-Efficient Knowledge Commons for AI Agents

Abstract

AI agents spend substantial compute searching for and synthesizing information, then discard most of what they find—even though it may be useful to other agents. We introduce AGENTPEDIA, an open knowledge commons in which agents deposit findings for reuse; agents may consult accumulated knowledge before recomputing it. Agents and humans may contribute, but we expect most readers to be agents, since the corpus does not prioritize human readability. AGENTPEDIA holds three kinds of contributable objects: content, as self-contained knowledge units or “finds” that retain evidence, validation, and applicability conditions; indices, which organize and score units for credibility, freshness, or task relevance and which any participant can build; and rules, which govern admission, reconciliation, validation, and reward, and which change through the same evidence-bearing process as the content they govern. Because the corpus can be flexibly reorganized, prior computation can be repurposed for tasks its producers did not anticipate, which knowledge held in model weights or private caches does not allow. Across 10 randomized task orders in a 50-task frozen-corpus evaluation, AGENTPEDIA reduces page reads by 32% and task-solving API tokens by 9% while raising answer completeness from 86% to 88%. In a higher-overlap setting where more prior work can be reused, page reads fall by 59% and API tokens by 21%, while completeness rises from 83% to 87%.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
AI agents, knowledge commons, cross-agent knowledge sharing, computational reuse, agent memory, retrieval, compute efficiency, collective intelligence

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_6uJyOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces AgentPedia, an open knowledge commons in which store agent findings for reuse. Three types of objects are stored in the accumulated knowledge: content as the knowledge units, indices that organize the units, and rules that govern the changing process. The knowledge can be produced in a flexible way, therefore, things from previous computation could be reused for later tasks. Across multiple various evaluation experiments, the proposed method is shown to reduce page reads and API costs, while also raising answer completeness.

Critical strengths
  • The proposed method is shown to preserve intermediate findings and reduce repeated research, which could provide some insights for future work in this direction.
  • The proposed approach is shown to provide benefits on reducing page access and API cost.
Critical weaknesses
  • The experiments and evaluation is limited, it only tests a domain-specific task, and it is unclear whether things can be generalized to different tasks and domains. There is also a lack of comparisons to related caching and retrieval methods.
  • There is a lack of more detailed analysis and ablation study on how different components and design choices of the proposed approach will influence the performance.
Questions for authors
  • How does the proposed approach compare with semantic caching and other retrieval methods over previous answers under the same total token budget?
  • How will the task relatedness influence the results? For example, if two tasks are more related, maybe the proposed approach might be more useful. Moreover, how about the case where there might contradicting information, will there be mechanisms to handle such scenarios?
  • Do benefits persist on naturally occuring tasks across domains and heterogeneous agent models? I think there should be more evaluation and analysis on more different settings.
Reviewer_nd6UOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes AgentPedia, a shared knowledge commons where AI agents deposit and reuse findings so that they do not recompute knowledge others have already found. It has three contributable layers:

  • Content: self-contained knowledge units ("finds") with evidence, scope, provenance, and state.
  • Indices: derived views such as credibility, freshness, and task lenses, which readers choose for retrieval.
  • Rules: versioned protocols for admission, reconciliation, validation, and rewards.

New units pass admission and reconciliation before publication, and their credibility is then maintained by randomly assigned peer validators. The paper also sketches incentive mechanisms (credits, payment, a router interface) and an evidence-based process for revising rules.

The experiments use 50 model-generated questions over a 91-page frozen corpus from MemoryArena, with GPT-5.6 Luna as the task solver. Compared with a no-persistence baseline, AgentPedia reduces page reads by 32% and task-solving tokens by 9%, and improves required-point coverage from 85.8% to 88.2% (averaged over 10 task orders). On paraphrased questions, the reductions are 59% and 21%. Validation, contributed indices, and rule revision are not evaluated.

Critical strengths
  1. Well-motivated and complete system design. The problem is real and growing: agents repeatedly spend compute to rediscover knowledge that others already found, and an open alternative to providers' private caches is a meaningful goal. The design goes beyond a shared cache and covers three levels:

    • The data level: units carry evidence, scope, and provenance.
    • The organization level: pluralistic, reader-selected indices, so quality is handled by indices rather than by gatekeeping.
    • The meta level: how the commons is governed and how its rules evolve.

    Several details are thoughtful, such as separating the evidence needed to publish from long-term credibility, and distinguishing "little scrutiny" from "contested." This makes the paper a useful blueprint for shared agent knowledge, although implementing it at scale would be challenging.

Critical weaknesses
  1. Most of the claimed design is not validated. The three-layer commons, the validated lifecycle, and the incentive and governance mechanisms are listed as contributions, but the experiments only test the content layer with the default index. Validation is disabled. Contributed indices, rule revision, and incentives are not evaluated. Admission and reconciliation outcomes are not reported. The claim that stored knowledge can be repurposed for tasks its producers did not anticipate is also untested. What is actually validated is essentially a structured shared cache, so most of the claimed contributions remain unsupported.
  2. The experimental setting is toy-scale. The setting has a single domain, 91 pages, 50 questions, and a final store of about 45 units. It uses one model with minimal reasoning. The tasks are easy: the baseline already reaches 86% coverage and reads only about 2.5 pages per task. In addition, a single agent solves the tasks sequentially, so the core scenario of many independent agents sharing a commons is never tested. The results do not show that the approach works in more diverse or realistic settings.
  3. There are no baselines or ablations. The only comparison is against no memory at all. The paper repeatedly argues that AgentPedia differs from semantic caching, RAG, and agent memory, but it does not compare against any of them, such as GPTCache-style caching of past answers or traces, ExpeL, G-Memory, cq, or the memory baselines in MemoryArena. There are also no ablations, for example removing the evidence and scope fields, the two-stage card selection, or reconciliation. It is therefore unclear whether the gains come from any form of caching or from the proposed design.
  4. The cost benefit is limited and may be negative. The headline 9% token saving excludes writeback. By the authors' own accounting in Table 6, writeback costs 401K tokens, so a single pass uses about 15% more tokens than the baseline, and break-even requires about 3.4 passes of comparable partial-overlap work. This estimate does not yet include the cost of validation, index maintenance, or reconciliation, which are the paper's core mechanisms and could add substantial overhead. In realistic, lower-overlap settings, it is doubtful that the approach is cost-effective.
Questions for authors
  1. Did admission and reconciliation actually run in the experiments, and how many candidates were rejected or merged? In Table 5, the third unit's date comes from the saved task answer rather than from the cited evidence. Does admission accept an agent's own previous answers as evidence?
  2. Under the same setting, how does AgentPedia compare with a simple semantic cache of past answers or traces? This comparison is needed to show that the proposed knowledge-unit design adds value beyond generic caching.
Reviewer_GYJyOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes AgentPedia, an open knowledge commons where AI agents deposit findings ("finds", with evidence, scope and provenance) for other agents to reuse. Indices and governance rules are also contributable objects. It describes admission, reconciliation, peer validation and incentive mechanisms. The evaluation compares an accumulating store with a no-persistence baseline on 50 questions over a 91-page frozen corpus. The authors report 32% fewer page reads, 9% fewer task-solving tokens and +2.4 pp answer completeness.

Critical strengths

It addresses a real cost, the compute wasted re-researching recurring questions, and it reports the cost of writeback honestly.

Critical weaknesses
  1. The experiments can't support the claims. For a proposal like this, the experimental evidence is everything, and here it is far too thin.
    • Scale: 50 questions in one domain, with one model (p.9).
    • What's actually tested: one agent reusing its own earlier findings on deliberately overlapping questions (p.8). The setting the paper motivates (different agents, different tasks, reuse "for tasks its producers did not anticipate", abstract) is never tested.
    • Effect sizes are small and overstated:
      • "−32% page reads" means 2.5 → 1.7 reads per task (Table 3), on tasks that need very little reading.
      • Reuse averages 0.8 events per task (p.8).
      • Quality rises only from 85.8 to 88.2 (Table 8). No significance test is reported, and the grader is the same model as the solver (p.10).
      • Once writeback is counted, total tokens increase on a single pass (Table 6, p.15).
    • No link between reuse and quality. Nothing shows that tasks using stored knowledge are answered better (Fig. 3).
  2. Most of the proposed system is unevaluated. Validation, contributed indices, incentives and rule revision aren't tested (p.8, p.14). Contributions 2 and 3 therefore rest on design description alone.
Questions for authors
  1. (Clarification) Is the commons meant for sharing across agents doing different tasks, or within related tasks? Can you show results with multiple agents and with non-overlapping tasks?
  2. (Clarification) Does reuse improve answer quality at the task level?
  3. (Clarification) How were the 50 evaluated questions selected from the 100 constructed (p.13)? Has the model-based grading been validated against humans?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.