acceptodds
Under review as a conference paper at ICLR 2027

A Claim-Centered Publication Framework for Efficient Auditing of AI-Assisted Theoretical Research

Abstract

AI systems are beginning to contribute directly to theoretical research and have already produced exciting mathematical results. As these systems generate new mathematical results at an increasing pace, a growing challenge is how to review these results efficiently. Existing projects release supporting artifacts organized around research runs rather than individual claims, which can make it difficult for reviewers to locate and assess the evidence supporting individual claims. We therefore propose a claim-centered publication framework designed to improve audit efficiency in AI-assisted theoretical research. Under this framework, a problem-specific AND/OR proof DAG serves as a research record maintained throughout the project and shared alongside the final paper. Each claim is represented by a node that records its dependencies, evidence, and review status and links directly to its supporting materials. Failed claims and proof routes are explicitly retained rather than discarded. We study two complementary cases in quantum query complexity and find that, in each, maintaining this record accounted for roughly 5% of the project’s total API-equivalent model cost. In our preliminary audit, the research-record condition used 20.9% fewer total tokens on average than the run-centered condition. The positive case, small-alphabet -Sum, yielded an lower bound for -Sum over odd cyclic alphabets of size , reducing the alphabet condition required by the previously used large-alphabet construction from to . The negative case, minimal-alphabet Set Equality, did not produce the desired explicit optimal-order adversary witness, but yielded an witness and reduced the remaining explicit construction to four unresolved proof obligations, conditional on a separately recorded supporting layer. Together, the two cases illustrate how the proposed framework can support efficient claim-level auditing through a research record that preserves intermediate results, failed routes, and unresolved questions for future research.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
AI-assisted theoretical research, claim-centered research records, mathematical reasoning, quantum query complexity, research auditing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kDDmOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to organize LLM outputs for Theoretical research in computer science or mathematics. The authors note that current AI-generated results in this area are usually shared in the form of a paper that contains only the ideas that worked, without necessarily sharing the ideas that did not work, the partial results towards a better or more general proof etc. This paper proposes to explicitly constrain the model to organize his research in the form of an AND/OR directed acyclic graph, which encodes the claims/lemmas/theorems used and the dependencies among those results. This additional structure can then be shared to help other people or AI agents to build on previous attempts without trying the same things over and over. The authors apply this idea to 2 separate problems in theoretical computer science. On these problems, they show that their approach induces only a small token overhead, while allowing for the AI models to find mistakes more reliably compared to just giving the flawed proof.

Critical strengths

-This paper studies a timely subject, given the recent results of AI generated proofs.

-The problems studied are interesting

Critical weaknesses

-The experiments are quite narrow on 2 specific problems. Results would be more compelling with more repetitions of this strategy on other problems.

-It is unclear what is precisely "paper + chat + evidence" compared to "paper + DAG + evidence" proposed by the authors. How is "paper + chat + evidence" constructed exactly ?

-Minor weakness: some sentence are really unclear, for instance in abstract "Existing projects release supporting artifacts organized around research runs rather than individual claims," was very unclear to me before reading the rest of abstract.

Questions for authors
  • Have you tried your framework on other problems ? Another thing that could be great is if your framework could be used to explain some of the recent math results by OpenAI more clearly. Have you tried your framework to explain some results?
Reviewer_6QdkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a new framework for the automated assessment and verification of the scientific significance and claims presented in research papers that contain theoretical results derived with the aid of AI. The framework centers on atomic, verifiable hypotheses and assertions, which are then organized into a directed acyclic graph (DAG) representing the paper as a whole. The authors argue that this approach is more effective and rigorous than verifying a paper based solely on specific runs of AI models. The study examines two concrete examples: one involving a successful proof of new theoretical results in the mathematics of quantum computing, and another involving an unsuccessful attempt. The framework itself relies on the repeated application of large language models (LLMs); in the paper’s appendices, the authors provide detailed explanations and instructions for implementing the framework.

Critical strengths

The article focuses on a new and highly relevant field that currently lacks an established methodology and is evolving in real time.

The article includes a wealth of practical material that facilitates the use of the proposed framework.

The verification methodology under discussion appears quite intuitive and sound. The proposed framework could prove useful as a practical tool.

Critical weaknesses

Although the article addresses a highly significant scientific issue, it is not entirely academic in nature. The proposed framework is currently described as a useful engineering solution rather than a scientific contribution; in effect, the paper presents only a proof-of-concept based on two specific examples. Neither the individual components of the framework nor the framework as a whole are validated or evaluated—neither automatically nor through expert assessment. The paper fails to adhere to an academic format: it lacks a literature review, and the core results are articulated quite vaguely. Within the field of NLP, there are several research areas closely related to the authors' work—notably fact-checking and agent-based frameworks for solving mathematical problems. It is worth noting that the authors, in their section on limitations, clearly outline the steps required to elevate the paper to a more rigorous academic standard. Furthermore, the attempt to integrate the description of the framework with the examples of mathematical problems is not particularly successful, making the main text difficult to follow. Without the appendix, grasping the essence of the underlying issues is extremely challenging. It might have been more effective to begin with some "toy" examples.

Questions for authors

What is the current stage of the research, and has any progress been made on the points listed under "Limitations"?

Reviewer_Xgj9OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a way to organize and publish the supporting material for AI-assisted theoretical research. The main object is a persistent AND/OR graph whose nodes represent mathematical claims and whose routes represent alternative arguments for those claims. Each node links to its evidence, dependencies, and review history. Failed arguments and unresolved claims are retained. The paper describes protocols for maintaining this record and illustrates the framework through two quantum query complexity projects: small-alphabet k-Sum and minimal-alphabet Set Equality. It also compares AI audits using the graph with audits using research transcripts or the paper alone.

Critical strengths

I think the problem addressed here is important. As AI systems produce more mathematical arguments, checking them becomes a serious bottleneck. Making it easier to locate the evidence for a particular claim, and to understand what depends on that claim, would be useful.

Critical weaknesses

It is not clear how well the result extends to other arguments. The evaluation also does not report incorrect objections raised by the auditors. It is also unclear how much the full framework adds beyond well-organized supporting material.

Questions for authors

I have two questions:

1.Were the graph and transcript conditions matched for the underlying mathematical information available to the auditor? How do the results change across repeated runs, and were incorrect objections assessed?

  1. How does the framework compare with a claim index or proof outline linked to the same evidence? Which features of the framework do the authors expect to account for a benefit beyond that simpler organization?

Comments on the paper

1 comment

Dear Reviewers,

The authors have responded to your comments. Could you please review their responses and update your reviews?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.