- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper focuses on chemical structure recognition and extraction from images. Instead of directly predicting the final chemical structure, it proposes ViCoR, a pipeline of iterative verification and revision. This is motivated by the observation that initial predictions often contain only minor errors compared with the ground truth.
The pipeline works as follows. The initial molecular graph prediction Y^0 is first rendered into an image. The coordinates also appear to be specified during rendering to align it with the input image. A verifier, implemented as a VLM, is then asked whether the two images represent the same structure. If they match, the prediction is returned. Otherwise, a reviser proposes local corrections using six types of node- and edge-level edits based on the input and rendered images (relates to Question 1 later). Although this is not explained clearly, my understanding from the prompt on page 15 is that the reviser produces editing instructions in a language or JSON format. An executor must therefore apply these instructions (related to Question 2 later). The process repeats until verification succeeds or the maximum number of revisions, T, is reached.
The verifier and reviser share a single VLM, which appears to be trained by the authors, although the training details are largely unclear in the main text (relates to Question 3 later). Finally, the authors conduct experiments and compare their approach with several existing methods.
Critical strengths
- The topic of the chemical structure extraction is well motivated and has been studied for a long time.
- Compared with the text content of the paper, the figures are more informative and help me a lot to understand the paper. However, some figures lack references from the main text, as noted later.
Critical weaknesses
- To be honest, I feel this manuscript is poorly prepared, particularly the method section. Several easy-found problems of mathematical writing, cross-referencing, and grammar are listed below. Setting aside the academic contributions of the paper, I believe that, in terms of the writing and structure of the manuscript itself, it is not yet ready for submission, let alone be accepted.
- Continuing the previous point, some important methodological details remain vague in the main content. After reading the method section, I found it difficult to follow the method and evaluate its soundness. I may have missed information in the appendix, but the main text does not provide sufficient context or clear references to appendix how these issues are addressed. I do not mean that every detail should be included in the main text. However, the authors should at least indicate that these questions have been considered and direct readers to the relevant appendix sections for further explanation. Meanwhile, repeated and redundant statements waste space that could be used to explain more important methodological details.
- I also have concerns that the method is technically unsound. Several important concepts or details are not clearly described, leaving many unresolved questions. Please see the questions below.
- Although code and checkpoints are not strictly required for submission, but I feel these are important, especially when it is hard to directly evaluate the results by only reading the paper. I believe it should be straightforward to provide code and checkpoints through an anonymous repository, and I hope the authors will do so. I would reconsider my assessment once the results are verified by myself; otherwise, I remain unconvinced by the reported results.
- Commercial tools such as Mathpix support optical chemical structure recognition and extraction as well. I believe comparisons with these tools are also useful information for providing readers more context of the positions of the methods with the commercial software.
Some issues of the academic writing:
-
Some of the mathematical writing is unclear, and the notation sometimes appears inconsistent or incorrect. The authors appear to formulate nearly every component mathematically, but these formulations do not seem to have been carefully considered and checked for clarity and consistency, making the method unnecessarily difficult to follow.
(a). In Eq. (10), is there a specific reason to introduce P? Why not directly write V = (I || I_tilde)? Similarly, why is P_idx needed in Eq. (11)? These additional symbols seem to complicate the presentation without a clear benefit.
(b). In Eq. (12), how many arguments does the reviser take, and what are they? The notation appears inconsistent even within the equation. Does the reviser receive only the prediction? My understanding is that it should receive both the prediction and the original input image.
(c). In Eq. (15), what are p_ver and p_rev? They are not explicitly defined. The reviser’s arguments also appear different from those used in the earlier equations. Please clarify these definitions and make the notation consistent.
-
Figures should be referenced in the main text. Figures 1 and 2 are not referenced in the main text, and Figure 2 is referenced only in the appendix. Explicitly referencing and discussing figures is a basic requirement of academic writing.
-
Displayed equations that are not referenced in the text are commonly left unnumbered. Although this is not a strict rule, it is a standard writing convention.
Typos and grammatical issues:
- Line 169: "an base" should be "a base."
- Line 169: "recognizer first the predicts" should be "recognizer first predicts."
- The sentence around Eq. (5) is incomplete.
Questions for authors
-
Some edits may require the coordinates of the location to be changed, such as when adding an atom. How does the VLM obtain these coordinates? Does it predict them directly from the image, or does it use additional tools? The prompt on page 15 does not appear to provide explicit coordinate information to the VLM.
-
How are the JSON-formatted editing instructions applied as actual modifications to the molecular graph? I assume that a rule-based executor performs these operations, but its operation is not clear from the main text.
-
The authors use only lines 291–300 to explain VLM training in the method section. How exactly is the model trained, and how are the training data constructed? After reading the method section, I still could not determine how training is performed. The experiments section briefly discusses synthetic data construction, but the explanation remains limited.
For example, how are the reviser's target edit sequences obtained? Are the corrections to erroneous predictions annotated by humans, or generated automatically (I think the synthetic data construction is targeting for this, right)? I think this is one of the most important aspects of the paper. If the appendix explains these details, the main text should explicitly indicate that these questions are addressed and direct readers to the relevant sections. The current organization and presentation make the training procedure difficult and struggling to understand.
I thank the authors for their responses. W1 (inconsistent of baselines in Tables 1 vs 2) is largely addressed. I recommend including GPT-4o in Table 2 as well, and more generally make the rows of Table 1 identical to the rows of Table 2 whenever possible.
Regarding W2 (the contribution is incremental), the main challenge in coordinate-preserving rendering is obtaining the coordinates and edges, after which one simply renders the atoms at the given coordinates and draws in the edges. This hard part is already provided by MolScribe. Similarly, the interface for index-anchored rendering is standard graph edit operations. Nevertheless, we appreciate the ablation studies showing that they do matter, so the observation that such a simple change is all that's needed for significantly improving verification could be considered interesting.
Overall, I raise my soundness score to 3 and overall score from 2 (weak rejection) to 3 (weak acceptance).