- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper presents AutoScientist, which uses a graph memory with typed relations and epistemic states to retain knowledge accumulated during research. The system alternates between literature acquisition, hypothesis generation, computational experiments, and memory consolidation.
Critical strengths
How research agents retain and use evidence over long investigations is a worthwhile problem.
Critical weaknesses
-
I would have liked to see more direct comparisons with existing research agents or complete memory systems. The current baselines mainly consist of dense retrieval, flat storage, and a PageRank variant, and the two standard memory benchmarks show no significant advantage over dense retrieval, leaving me unsure what specific improvements this relatively complicated design brings.
-
I found the paper difficult to follow. The abstract is unusually long and packs in too many results and strong claims, while the introduction quickly jumps to the contribution list. It does not clearly explain where existing methods fall short or what specific difficulty this method solves. The contributions also read more like a list of experimental results.
Questions for authors
-
Are direct comparisons with complete structured-memory systems or research agents available? If so, could you point to the results and clarify whether the models and compute budgets are matched?
-
Which comparisons support the abstract’s claim of matching frontier commercial systems? Are the question subsets, context budgets, and evaluation procedures identical?