- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Thoth, a theorem-oriented parser for planar geometry diagrams. It converts images into a Theorem-oriented Formal Language (TOFL) that represents geometric primitives, scene-applicable theorem instances, and question-relevant theorems, associating theorems with concrete geometric objects through Bind() and Support(). The authors construct the GT-210K synthetic dataset with 90 theorem templates. Training begins with supervised learning of scene-level parsing, followed by question-relevant theorem selection using labels extracted from reference solutions to real geometry problems. GRPO then optimizes rewards for binding, reconstruction, and geometric constraints. The resulting structured information is provided as auxiliary context to frozen downstream reasoning models. Experiments evaluate primitive parsing, theorem recognition and binding, and answer accuracy on four geometry QA benchmarks. The results support the representation's utility for downstream reasoning in most of the tested model–dataset combinations.
Critical strengths
-
The problem formulation and representation design are valuable. The paper distinguishes identifying diagram objects, determining applicable knowledge, and performing deduction, while further separating scene-applicable theorems from question-relevant ones. TOFL specifies object roles, supporting primitives, premises, and conclusions in addition to theorem names, providing downstream models with concrete theorem instances and their associations with diagram objects.
-
GT-210K provides a scalable source of supervision for jointly learning theorem knowledge and visual objects. The data engine constructs diagrams according to theorem premises, checks for other theorem instances supported by the resulting scene, and includes hard negatives that violate key premises. This design covers theorem co-occurrence and potentially confusing configurations, making it well suited to training primitive recognition and theorem applicability prediction.
-
The ablations and downstream experiments provide complementary evidence. In Table 4, the SFT model trained with full TOFL improves Primitive F1 from 0.7464 to 0.7942 relative to the primitive-only variant, and scene theorem F1 from 0.7210 to 0.8431 relative to the theorem-name-only variant. In Figure 4(b), the QA accuracy gain for Qwen3-VL-8B on MathVerse increases from 1.55 percentage points with primitives alone to 4.04 percentage points with the full information. Figure 4(c) also shows that selecting question-relevant theorems is more effective than directly providing all scene-applicable theorems. These results support the practical value of the joint representation and question-conditioned selection, although the benefits depend on the downstream model and dataset, with some combinations in Table 3 showing no improvement.
Critical weaknesses
-
Direct evidence of the core theorem-binding capability on real geometry problems remains limited. In Section 4.1 and Table 2, scene theorem F1 and binding accuracy are evaluated on 600 synthetic diagrams, whereas quantitative theorem evaluation on real diagrams consists of theorem-name selection F1 on 200 samples. The latter evaluates question-relevant theorem selection but does not directly test whether the theorems are bound to the correct objects and their premises are supported by the diagram or problem statement. The downstream QA improvements in Table 3 and the example in Figure 17 provide positive evidence, but do not quantify binding reliability on real diagrams. Manually checking the joint correctness of theorem names, object bindings, and premises on a small set of real problems would more directly support the paper's central claim.
-
The mechanism by which the theorem-local reconstruction reward supports binding needs clarification. Lines 257–261 state that incorrect bindings distort local reconstructions. However, in Eq. (8), h is drawn from the ground-truth theorem set, and both the predicted and reference renderings use the same ground-truth support set . If the renderer simply retrieves predicted primitives using the identifiers in this set, changing only the predicted
Bind()/Support()while keeping the predicted primitives fixed would leave both the local reconstruction and unchanged. Such a reward could still help theorem prediction indirectly by improving local geometry, and this does not undermine the explicit penalty for incorrect bindings provided by . Nevertheless, the direct sensitivity to incorrect bindings described in the text is not apparent in the equation. Figure 4(a) also ablates the global and local reconstruction rewards together without separating their effects. Please clarify whether local rendering additionally depends on the predicted theorem's bindings or support set, or whether the intended benefit is indirect through improved local geometry. -
There may be minor formatting errors: the Scene and Question legend entries in Figure 4(c) may be missing a "+" sign to match +Primitive, and there appears to be an extra space or indentation before "Here" on line 172. Please check these details and make the formatting consistent.
Questions for authors
-
Could you report the joint correctness of theorem names, object bindings, and premise support from the diagram or problem statement on a small set of real geometry problems, distinguishing theorem-selection errors from object-binding errors? This would help assess how grounding learned on synthetic diagrams transfers to real problems.
-
In Eq. (8), does local rendering on the prediction side depend on the predicted theorem's
Bind()/Support()in addition to the ground-truth support set ? Does change whenBind()/Support()is modified while the predicted primitives remain fixed, or does this reward primarily help binding indirectly through improved local geometry? A simple perturbation example would help clarify the mechanism.
4 comments
Official Comment by Authors (2/2)
W.2 / Q.2 Mechanism of theorem-local reconstruction reward in improving theorem–primitive binding
We thank for this important question and apologize for the imprecise wording in our original manuscript.
We clarify that the theorem-local reconstruction reward primarily improves binding indirectly by refining theorem-relevant local geometry. Specifically, we use the primitive IDs from the ground-truth Support() to construct local subdiagrams using predicted and ground-truth primitive attributes (e.g., point coordinates), and compute their reconstruction similarity as the reward.
Our intention is to encourage the model to better understand the relationships between theorems and their supporting local geometric structures by explicitly supervising the reconstruction of theorem-relevant subdiagrams. This indirectly enhances theorem–primitive grounding / binding through self-supervised reconstruction, while also improving overall primitive prediction. As suggested, we further validate these benefits through separate ablations of the local and global reconstruction rewards:
| Primitive F1 ↑ | TED ↓ | Syn. MSE ↓ | Syn. SSIM ↑ | Syn. DINO ↑ | Real MSE ↓ | Real SSIM ↑ | Real DINO ↑ | Scene Thm. F1 ↑ | Binding Acc. ↑ | Selected Thm. F1 ↑ | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ✗ | ✗ | 0.8142 | 0.1328 | 0.0120 | 0.9376 | 0.9589 | 0.0372 | 0.7921 | 0.7231 | 0.8429 | 0.8842 | 0.8598 |
| ✗ | ✓ | 0.8237 | 0.1201 | 0.0085 | 0.9450 | 0.9637 | 0.0364 | 0.8002 | 0.7659 | 0.8572 | 0.8957 | 0.8611 |
| ✓ | ✗ | 0.8294 | 0.1192 | 0.0092 | 0.9608 | 0.9699 | 0.0359 | 0.7983 | 0.7730 | 0.8570 | 0.9102 | 0.8637 |
| ✓ | ✓ | 0.8384 | 0.1198 | 0.0068 | 0.9708 | 0.9750 | 0.0345 | 0.8139 | 0.7982 | 0.8591 | 0.9219 | 0.8634 |
The results show that:
- Introducing improves theorem binding accuracy by 2.60 percentage points (notably higher than the 1.15-point improvement from ), indicating that it indeed indirectly improves binding accuracy.
- Moreover, generally brings greater improvements than across most metrics, benefiting from its fine-grained supervision.
- Adding on top of further improves reconstruction fidelity and primitive prediction. We attribute this improvement to more accurate theorem-relevant local geometry understanding, which provides more reliable theorem-level visual evidence and facilitates learning theorem–primitive binding.
We have also revised the original description of the mechanism to avoid potential misunderstandings. Thank you again for your careful reading and thoughtful comment, which are very valuable in improving the quality of our final manuscript.
W.3 Minor formatting errors
We have added the missing "+" in Figure 4(c) and removed the extra space at the beginning of Line 172 in the updated manuscript. Thank you for your careful reading and for pointing out these issues. We have also carefully reviewed the remaining text and figures to ensure that no similar issues remain.
We would like to thank the Reviewer once again for your careful reading and valuable feedback, which have helped us improve the evaluation and presentation of our work. We hope that our revisions have fully addressed your concerns.
Thank you for the clarifications and additional experiments. My concerns have been addressed, and I will maintain my current score.
Esteemed Reviewer,
Thank you for your response and for recommending acceptance.
We sincerely appreciate your time and thoughtful consideration of our submission. Meantime, if there is anything else that we can improve or answer better, do not hesitate to let us know.
Best regards, The Authors
Official Comment by Authors (1/2)
We thank the Reviewer for recoginizing the contribution of our work. Please find our detailed answers below.
W.1 / Q.1 Qualitative results of theorem binding on real QA diagrams
This is a very valuable suggestion. Following the Reviewer's recommendation, we manually evaluate the predicted theorem names and binding information on 100 randomly selected real geometry problem diagrams (50% of the total test set):
For the 100 selected samples, we further categorize the errors to distinguish theorem-selection errors from object-binding errors:
All metrics are computed following the same protocols as our original evaluation. Following the Reviewer's suggestion, we additionally introduce Premises Acc., which measures whether the conditions specified in the predicted
Premise()statements are correct. Joint Acc. measures the proportion of predicted theorem instances for which the theorem name, object binding, and premises are all correct. Note that each sample may contain multiple question-relevant theorems.The results demonstrate that Thoth maintains strong generalization performance on real QA diagrams, achieving a binding accuracy comparable to that on synthetic diagrams. This is consistent with our design objective of learning primitive–theorem understanding from synthetically generated diagrams with fine-grained annotations and transferring this capability to real-world diagrams. Moreover, we observe that binding errors are rare once the correct theorem is identified, suggesting that accurate theorem recognition is strongly associated with correct primitive binding.