- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
When an agentic scientist improves a score, it's unclear if they did so meaningfully or gamed the evaluator. The authors claim to provide an audit to disentangle these cases, and I am guessing that they do this in the domain of RNA topology. The authors produce a list of six results about their audit, but do not provide any takeaways and claim that they cannot conclusively say that there was a "pass" or "fail" verdict in the audit.
Critical strengths
- None, this paper should be desk rejected for abusing AI systems in the research process
Critical weaknesses
-
This paper is impossible to understand, so I cannot even begin to judge it. I think I was lost immediately after the first paragraph of the introduction. The authors use terms like "RNA inverse design", "Nested secondary structures", "context-free descriptions", and "seudoknot-free folding oracle" without defining them, citing papers that discuss them, or connecting their relevance to the research question. This paper needs a substantial rewrite. I would recommend the authors follow a more standard flow of: Provide context on the problem -> discuss related work -> introduce the contribution or hypotheses -> discuss experimental setup -> provide results, discussion, and takeaway. Even Figure 1 is very clearly AI-generated. There are two explanations for this: 1) either I am extremely unfamiliar to this field; or 2) the paper uses extensive AI writing which makes it impossible to understand
-
The experimental results seem extremely weak. There are only 60 examples used for evaluation. I am not an expert on the field of RNA topology, but it is not discussed whether this is a small-scale experiment or what we would expect from this domain
Questions for authors
- It seems like there is another paper by the same authors under review at ICLR 2027 which is cited four times:
“solved” is a choice of verifier, not a fact: Deterministic cost–robustness frontiers of verifier choice in RNA and SAT
I would recommend the ACs to look into this paper as well, but from the title it seems like the domain and experiment setup could be very similar. Can the authors explain the contributions of this paper beyond that other ICLR paper?