acceptodds
Under review as a conference paper at ICLR 2027

AutoScientist: An Autonomous Agent for Long-Horizon Scientific Discovery via Connectome Memory

Abstract

Accelerating the rate of scientific discovery requires autonomous agents capable of continuous literature assimilation, rigorous hypothesis generation, and empirical verification over extended horizons. Existing autonomous research agents rely on linear flat-file scratchpads or unindexed vector databases for memory. These representations suffer from context window saturation, catastrophic forgetting of early observations, and an inability to detect contradictions across multihop causal chains. To resolve these limitations, we introduce AutoScientist, an autonomous discovery system organized around a biologically inspired, multirelational graph connectome. AutoScientist treats memory as an active graph, modelling scientific knowledge as an evolving network of typed concepts, causal relations, and empirical findings whose structure persists as it grows. The architecture executes diurnal discovery cycles: active phases acquire literature, formulate structured hypotheses, and run computational simulations in code sandboxes, while nocturnal phases apply multi-pass consolidation algorithms that resolve logical contradictions, prune unreinforced associations, and synthesize structural analogies. We evaluate AutoScientist across 20 longitudinal discovery cycles and three disparate scientific domains: non-equilibrium statistical mechanics, systems biology gene regulatory circuits, and non-binary error-correcting codes. Empirical evaluations show that the agent’s memory develops a sparse, modular small-world topology that no degree-matched random null reproduces. In controlled multihop retrieval and falsification benchmarks, graph memory holds every chain step and resolves every recorded contradiction, whereas dense vector retrieval recovers only a third of a chain’s interior, never exceeds chance on a recorded contradiction, and fills nearly half of its retrieved context with decoys.A sweep of workingmemory capacity shows that bounded focus strikes an optimal balance between prompt efficiency and thrashing avoidance, and neuromodulatory state transitions prevent stagnation on intractable problems. The full agent runs on a single locally hosted 32B open-weight model, and matches the suite scores reported for frontier commercial systems on the same long-memory sets, so strong long-horizon scientific discovery is within reach of modest, fully local hardware. These results establish connectome-based memory as an effective computational foundation for long-horizon autonomous scientific research.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
connectome memory, graphs, autonomous research agents, multi-hop reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_JGP7OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents AutoScientist, which uses a graph memory with typed relations and epistemic states to retain knowledge accumulated during research. The system alternates between literature acquisition, hypothesis generation, computational experiments, and memory consolidation.

Critical strengths

How research agents retain and use evidence over long investigations is a worthwhile problem.

Critical weaknesses
  1. I would have liked to see more direct comparisons with existing research agents or complete memory systems. The current baselines mainly consist of dense retrieval, flat storage, and a PageRank variant, and the two standard memory benchmarks show no significant advantage over dense retrieval, leaving me unsure what specific improvements this relatively complicated design brings.

  2. I found the paper difficult to follow. The abstract is unusually long and packs in too many results and strong claims, while the introduction quickly jumps to the contribution list. It does not clearly explain where existing methods fall short or what specific difficulty this method solves. The contributions also read more like a list of experimental results.

Questions for authors
  1. Are direct comparisons with complete structured-memory systems or research agents available? If so, could you point to the results and clarify whether the models and compute budgets are matched?

  2. Which comparisons support the abstract’s claim of matching frontier commercial systems? Are the question subsets, context budgets, and evaluation procedures identical?

Reviewer_s1u3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This is a solid, well-engineered, and clearly motivated piece of work. It proposes a long-horizon autonomous scientific discovery agent called connectome‑memory, based on a typed, signed weighted knowledge graph. It features an awake‑sleep circadian cycle design, where the sleep phase performs multi-step memory consolidation and simulates synaptic weight decay to prune the knowledge network. The entire system runs entirely on a local open-source model. The authors designed specialized memory probes and also validated the system using two public long-memory benchmarks, three scientific domain tasks, and numerous ablation studies.

Critical strengths
  1. Focused on the real pain points of current autonomous research agents' memory: flat logs and dense vector retrievals can't naturally represent/support contradictory epistemic relationships, multi-step causal chains are prone to breaking, retrieval often mixes in decoys and information gets overwhelmed during long-term operation.
  2. Modeled memory as a directed type graph that can be modified and evolved in place, distinguishing 7 types of nodes and 7 types of relationship edges; separated active reasoning and memory consolidation across day-night cycles; introduced weight decay to simulate synaptic pruning; added neural modulation mechanisms like dopamine and frustration, triggering analogy-based incubation when the agent hits a deadlock to prevent task stagnation.
  3. The paper employs a comprehensive set of evaluation dimensions:
    • Developed a custom memory probe to specifically measure multi-hop chain recall, contradiction resolution, and interference filtering;
    • Conducted end-to-end scientific discovery experiments across 3 different scientific domains and set notebook flat files as a strong baseline;
    • Performed various ablations;
    • Used dual evaluators, both strict and lenient scoring, manual audits, and statistical hypothesis tests, providing a solid basis for reproducibility.
Critical weaknesses
  1. Although the term "connectome memory" is used extensively throughout the title and abstract, the authors acknowledge multiple times in the main text that the system is neither a computational model of a biological brain's connectome nor does it align with the statistical characteristics of real biological neural networks.

  2. A core performance discrepancy remains inadequately explained: there is a significant gap between the authors' custom evaluation metrics and the results from public benchmarks. This constitutes the paper's most critical flaw:

  • In the authors' own memory tests, graph-based memory vastly outperforms dense vector retrieval; the system achieves near-perfect scores on multi-hop chain reasoning and conflict-resolution tasks, remaining virtually unaffected by interference.
  • Conversely, on general long-term memory benchmarks such as LongMemEval-S, graph-based memory shows no statistically significant difference from iterative dense retrieval. The authors dismiss this by merely suggesting that "general benchmarks tend to assess generative capabilities rather than memory capabilities".
  1. The appendix contains a key, counterintuitive experimental finding that is not mentioned in the main text. The ablation study in Appendix B reveals that when the system resolves conflicts during reasoning, it relies on the "epistemic status" field internal to each node, rather than on explicit "conflict" edges. Removing or inverting these "conflict" edges has no impact on the system's performance; only by altering the internal state of the nodes does performance plummet to the level of random guessing.

  2. The evaluation metrics for end-to-end agent experiments are ambiguous, and the interpretation of the baselines is potentially misleading. The paper presents two sets of evaluation criteria: (1) a fixed number of iterations, and (2) a fixed wall-clock time. While full data is provided in Appendix H, the main text fails to offer an adequate explanation for the discrepancies between these results.

  3. The system has a clear performance ceiling, constrained by the generative capabilities of the large language model (LLM)—a point that has yet to be fully explored. Experiments conducted on DiscoveryBench reveal that when a task’s bottleneck lies in generative reasoning rather than memory retrieval, there is no significant performance difference between graph-based memory, dense vectors, and flat storage structures. Furthermore, models frequently generate fictitious numerical results.

  4. Existing graph-memory agent systems, such as GraphRAG, HippoRAG, and A-Mem, differ from AutoScientist in the following ways:

  • They primarily rely on static document-to-graph construction, whereas AutoScientist dynamically modifies, prunes, and merges the graph in-situ during agent execution.
  • Existing systems rarely treat "contradiction" as a first-class relationship type; they typically focus on storing associations and supporting evidence, failing to explicitly model conflicting evidence. These two aspects represent the core innovations of the paper; however, the "Related Work" section fails to clearly articulate or contrast these points.
Questions for authors
  1. The perturbation ablation experiments in Appendix B show that the performance of resolving contradictions depends on the epistemic status field inside nodes, not on CONTRADICTS type edges. But both the abstract and the main text treat explicit contradiction edges as a core design. Can you explain:
  • What function do CONTRADICTS edges actually serve in the system?
  • Since removing contradicts edges doesn’t impair contradiction resolution, what’s the necessity of having these edges?
  • Why doesn’t the main text report this key experimental finding at all? Could this mislead readers about how the system works?
  1. In your custom memory probes, the connectome memory performs much better than dense vector retrieval; yet on general long-memory benchmarks like LongMemEval-S and LoCoMo-10, graph memory and dense retrieval perform roughly the same statistically.
  • What’s the root cause of this huge divergence between the two evaluation results?
  • If general benchmarks can’t show a gain, how can you prove that the benefits of graph memory can generalize to real scientific tasks, and not just the handcrafted probe effects?
  1. The end-to-end experiments have two sets of metrics. After 4 cycles, the notebook baseline almost stops producing sandbox-runnable valid hypotheses. Please explain the mechanism behind this. The main text makes it easy for readers to think that connectome is completely superior in throughput. How do you plan to revise the text narrative to avoid this misunderstanding?
Reviewer_aideOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies memory maintenance for long-horizon autonomous scientific research. The proposed AutoScientist combines typed graph memory with a wake-sleep cycle for research and memory consolidation. Evaluation covers memory probes, standard memory benchmarks, and scientific missions in three domains. The design is interesting, although stronger evidence of downstream discovery benefits and more controlled comparisons would strengthen the paper.

Critical strengths
  1. Meaningful motivation. Maintaining findings and their relationships over extended research processes is an important problem for autonomous scientific agents.
  2. Targeted technical design. The typed graph explicitly records claims, evidence, and contradictions, while consolidation updates the memory over time. These components form a coherent solution to the studied problem.
  3. Detailed experimental analysis. The paper examines several aspects of memory, compares retrieval methods, and assesses evaluation sensitivity using a second judge and a human audit. The use of a locally hosted open-weight model is also practically valuable.
Critical weaknesses
  1. Limited evidence of improved scientific discovery. The method does not significantly outperform dense retrieval on the standard memory suites, and the DiscoveryBench results do not establish a clear advantage. Producing more experiments during a mission does not by itself demonstrate more correct or valuable discoveries.
  2. Insufficiently controlled system comparisons. The notebook baseline is matched by cycle count but is substantially faster per cycle. Comparisons under matched computational budgets and with complete structured-memory systems would help establish the contribution beyond the existing shared-store PageRank control.
  3. Incomplete attribution to the proposed mechanisms. Contradiction-probe accuracy remains unchanged after deleting contradiction edges, while the consolidation replay disables LLM calls. Further controls are needed to establish what typed relations and generative consolidation contribute to downstream performance.
Questions for authors

Can you give more experimental evidence on how the proposed method improve scientific discovery?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.