- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes an evaluation protocol for scientific agents, the agent commits to a belief, a prediction, a falsifier, and an action. this commitment is hashed then to make sure it's irreversible. It then receives only the evidence from the chosen action and returns a revised belief together with a proposal on what to do next. another component checks for the correctness of this flow.
Critical strengths
- the motivation is on time and there is indeed need for making sure that what agents propose is actually grounded in real, correct evidences
Critical weaknesses
-
The paper is very hard to read. The abstract packs in numbers whose meaning depends on values defined much later. Almost every result sits among several qualifications. Version-history details also appear in the paper, for example “V3.1 displayed 14.4,” the salt mismatch, and recovered-script rounding differences. That material doesn't belongs in a conference submission. The main text spends more space explaining what each result does not show (e.g. this is this not that) than what it does show. The introduction is too small and it doesn't give enough motivation + clear explanation of current challenges and the solution, it just jumps to next section.
-
The novelty is low, the paper explicitly disclaims new hashing, timestamping, preregistration. It also states that ACEC “supplies the binding and admission mechanism,” other works like Preregistered Belief Revision Contracts (Alqithami, 2026) already specify evidence triggers, admissible revision, and verifiable evidence. ResearchLoop (Xia & Wang, 2026) already provides evidence-gated admission rules, it doesn't explicitly show what is novel in the paper and the contributions seems to not be enough
Questions for authors
- How would you validate falsifiers?
- How would you make sure the distribution returned as belief distribution is actually correctly formalized by the agent?