acceptodds
Under review as a conference paper at ICLR 2027

Thoth: Theorem-oriented Parsing of Planar Geometry Diagrams with MLLMs

Abstract

When solving geometry problems, perceiving visual primitives is only the first step; identifying relevant theorems provides essential knowledge for subsequent reasoning. We formulate this process as , then , and finally . Current MLLMs often entangle the latter two within lengthy reasoning trajectories and struggle to identify relevant theorems, leaving subsequent deduction susceptible to unsupported assumptions. We present Thoth, a planar geometry diagram parser that provides both perceived geometric and question-relevant to guide subsequent deeper . To align visual perception with theorem knowledge, we introduce a Theorem-oriented Formal Language (TOFL) and construct GT-210K, a large-scale synthetic dataset with TOFL annotations for scalable supervised fine-tuning. We then perform reinforcement learning with fine-grained process reward signals to further ground theorem predictions in supporting visual evidence and problem-specific constraints. Experiments show that Thoth achieves strong theorem grounding while improving primitive prediction. Across four geometry VQA benchmarks, the grounded primitives and theorems effectively improve downstream reasoning performance, demonstrating their value as explicit knowledge for subsequent deduction. Code, datasets, and models will be publicly available upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
MLLMs, structured diagram parsing, plane geometry

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_pd6oOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Thoth, a theorem-oriented parser for planar geometry diagrams. It converts images into a Theorem-oriented Formal Language (TOFL) that represents geometric primitives, scene-applicable theorem instances, and question-relevant theorems, associating theorems with concrete geometric objects through Bind() and Support(). The authors construct the GT-210K synthetic dataset with 90 theorem templates. Training begins with supervised learning of scene-level parsing, followed by question-relevant theorem selection using labels extracted from reference solutions to real geometry problems. GRPO then optimizes rewards for binding, reconstruction, and geometric constraints. The resulting structured information is provided as auxiliary context to frozen downstream reasoning models. Experiments evaluate primitive parsing, theorem recognition and binding, and answer accuracy on four geometry QA benchmarks. The results support the representation's utility for downstream reasoning in most of the tested model–dataset combinations.

Critical strengths
  1. The problem formulation and representation design are valuable. The paper distinguishes identifying diagram objects, determining applicable knowledge, and performing deduction, while further separating scene-applicable theorems from question-relevant ones. TOFL specifies object roles, supporting primitives, premises, and conclusions in addition to theorem names, providing downstream models with concrete theorem instances and their associations with diagram objects.

  2. GT-210K provides a scalable source of supervision for jointly learning theorem knowledge and visual objects. The data engine constructs diagrams according to theorem premises, checks for other theorem instances supported by the resulting scene, and includes hard negatives that violate key premises. This design covers theorem co-occurrence and potentially confusing configurations, making it well suited to training primitive recognition and theorem applicability prediction.

  3. The ablations and downstream experiments provide complementary evidence. In Table 4, the SFT model trained with full TOFL improves Primitive F1 from 0.7464 to 0.7942 relative to the primitive-only variant, and scene theorem F1 from 0.7210 to 0.8431 relative to the theorem-name-only variant. In Figure 4(b), the QA accuracy gain for Qwen3-VL-8B on MathVerse increases from 1.55 percentage points with primitives alone to 4.04 percentage points with the full information. Figure 4(c) also shows that selecting question-relevant theorems is more effective than directly providing all scene-applicable theorems. These results support the practical value of the joint representation and question-conditioned selection, although the benefits depend on the downstream model and dataset, with some combinations in Table 3 showing no improvement.

Critical weaknesses
  1. Direct evidence of the core theorem-binding capability on real geometry problems remains limited. In Section 4.1 and Table 2, scene theorem F1 and binding accuracy are evaluated on 600 synthetic diagrams, whereas quantitative theorem evaluation on real diagrams consists of theorem-name selection F1 on 200 samples. The latter evaluates question-relevant theorem selection but does not directly test whether the theorems are bound to the correct objects and their premises are supported by the diagram or problem statement. The downstream QA improvements in Table 3 and the example in Figure 17 provide positive evidence, but do not quantify binding reliability on real diagrams. Manually checking the joint correctness of theorem names, object bindings, and premises on a small set of real problems would more directly support the paper's central claim.

  2. The mechanism by which the theorem-local reconstruction reward supports binding needs clarification. Lines 257–261 state that incorrect bindings distort local reconstructions. However, in Eq. (8), h is drawn from the ground-truth theorem set, and both the predicted and reference renderings use the same ground-truth support set S(h)S(h). If the renderer simply retrieves predicted primitives using the identifiers in this set, changing only the predicted Bind()/Support() while keeping the predicted primitives fixed would leave both the local reconstruction and rlocalr_{local} unchanged. Such a reward could still help theorem prediction indirectly by improving local geometry, and this does not undermine the explicit penalty for incorrect bindings provided by rbindr_{bind}. Nevertheless, the direct sensitivity to incorrect bindings described in the text is not apparent in the equation. Figure 4(a) also ablates the global and local reconstruction rewards together without separating their effects. Please clarify whether local rendering additionally depends on the predicted theorem's bindings or support set, or whether the intended benefit is indirect through improved local geometry.

  3. There may be minor formatting errors: the Scene and Question legend entries in Figure 4(c) may be missing a "+" sign to match +Primitive, and there appears to be an extra space or indentation before "Here" on line 172. Please check these details and make the formatting consistent.

Questions for authors
  1. Could you report the joint correctness of theorem names, object bindings, and premise support from the diagram or problem statement on a small set of real geometry problems, distinguishing theorem-selection errors from object-binding errors? This would help assess how grounding learned on synthetic diagrams transfers to real problems.

  2. In Eq. (8), does local rendering on the prediction side depend on the predicted theorem's Bind()/Support() in addition to the ground-truth support set S(h)S(h)? Does rlocalr_{local} change when Bind()/Support() is modified while the predicted primitives remain fixed, or does this reward primarily help binding indirectly through improved local geometry? A simple perturbation example would help clarify the mechanism.

4 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/2)

We thank the Reviewer for recoginizing the contribution of our work. Please find our detailed answers below.


W.1 / Q.1 Qualitative results of theorem binding on real QA diagrams

This is a very valuable suggestion. Following the Reviewer's recommendation, we manually evaluate the predicted theorem names and binding information on 100 randomly selected real geometry problem diagrams (50% of the total test set):

EvaluationSynthetic TotalReal TotalReal 50%
Theorem F10.85910.86340.8753
Binding Acc.0.9219--0.8949
Premises Acc.0.8682--0.8524
Joint Acc.0.7982--0.7629

For the 100 selected samples, we further categorize the errors to distinguish theorem-selection errors from object-binding errors:

Error TypePercentage
theorem selection error (missed)18.0%
Extra predicted theorems (unmatched to any GT)32 instances
Object binding error (given a theorem name match)10.5%

All metrics are computed following the same protocols as our original evaluation. Following the Reviewer's suggestion, we additionally introduce Premises Acc., which measures whether the conditions specified in the predicted Premise() statements are correct. Joint Acc. measures the proportion of predicted theorem instances for which the theorem name, object binding, and premises are all correct. Note that each sample may contain multiple question-relevant theorems.

The results demonstrate that Thoth maintains strong generalization performance on real QA diagrams, achieving a binding accuracy comparable to that on synthetic diagrams. This is consistent with our design objective of learning primitive–theorem understanding from synthetically generated diagrams with fine-grained annotations and transferring this capability to real-world diagrams. Moreover, we observe that binding errors are rare once the correct theorem is identified, suggesting that accurate theorem recognition is strongly associated with correct primitive binding.

AuthorsOpenReview ↗

Official Comment by Authors (2/2)

W.2 / Q.2 Mechanism of theorem-local reconstruction reward in improving theorem–primitive binding

We thank for this important question and apologize for the imprecise wording in our original manuscript.

We clarify that the theorem-local reconstruction reward primarily improves binding indirectly by refining theorem-relevant local geometry. Specifically, we use the primitive IDs from the ground-truth Support() to construct local subdiagrams using predicted and ground-truth primitive attributes (e.g., point coordinates), and compute their reconstruction similarity as the reward.

Our intention is to encourage the model to better understand the relationships between theorems and their supporting local geometric structures by explicitly supervising the reconstruction of theorem-relevant subdiagrams. This indirectly enhances theorem–primitive grounding / binding through self-supervised reconstruction, while also improving overall primitive prediction. As suggested, we further validate these benefits through separate ablations of the local and global reconstruction rewards:

rlocalr_{\mathrm{local}}rglobalr_{\mathrm{global}}Primitive F1 ↑TED ↓Syn. MSE ↓Syn. SSIM ↑Syn. DINO ↑Real MSE ↓Real SSIM ↑Real DINO ↑Scene Thm. F1 ↑Binding Acc. ↑Selected Thm. F1 ↑
✗✗0.81420.13280.01200.93760.95890.03720.79210.72310.84290.88420.8598
✗✓0.82370.12010.00850.94500.96370.03640.80020.76590.85720.89570.8611
✓✗0.82940.11920.00920.96080.96990.03590.79830.77300.85700.91020.8637
✓✓0.83840.11980.00680.97080.97500.03450.81390.79820.85910.92190.8634

The results show that:

  • Introducing rlocalr_{\mathrm{local}} improves theorem binding accuracy by 2.60 percentage points (notably higher than the 1.15-point improvement from rglobalr_{\mathrm{global}}), indicating that it indeed indirectly improves binding accuracy.
  • Moreover, rlocalr_{\mathrm{local}} generally brings greater improvements than rglobalr_{\mathrm{global}} across most metrics, benefiting from its fine-grained supervision.
  • Adding rlocalr_{\mathrm{local}} on top of rglobalr_{\mathrm{global}} further improves reconstruction fidelity and primitive prediction. We attribute this improvement to more accurate theorem-relevant local geometry understanding, which provides more reliable theorem-level visual evidence and facilitates learning theorem–primitive binding.

We have also revised the original description of the mechanism to avoid potential misunderstandings. Thank you again for your careful reading and thoughtful comment, which are very valuable in improving the quality of our final manuscript.

W.3 Minor formatting errors

We have added the missing "+" in Figure 4(c) and removed the extra space at the beginning of Line 172 in the updated manuscript. Thank you for your careful reading and for pointing out these issues. We have also carefully reviewed the remaining text and figures to ensure that no similar issues remain.


We would like to thank the Reviewer once again for your careful reading and valuable feedback, which have helped us improve the evaluation and presentation of our work. We hope that our revisions have fully addressed your concerns.

Thank you for the clarifications and additional experiments. My concerns have been addressed, and I will maintain my current score.

Esteemed Reviewer,

Thank you for your response and for recommending acceptance.

We sincerely appreciate your time and thoughtful consideration of our submission. Meantime, if there is anything else that we can improve or answer better, do not hesitate to let us know.

Best regards, The Authors

Reviewer_5z2kOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

To overcome Multimodal Large Language Models' tendency to focus on low-level visual primitives while ignoring high-level mathematical deduction in geometry, this paper introduces TOFL (Theorem-oriented Formal Language), a structured representation that explicitly binds visual elements to both scene-applicable and question-relevant theorems. Using this representation, the authors create the GT-210K synthetic dataset and train Thoth, an MLLM optimized via two-stage Supervised Fine-Tuning and Reinforcement Learning with custom rewards for theorem grounding, visual reconstruction, and geometric constraint satisfaction. Validation shows that Thoth dramatically outperforms existing models (including GPT-5.6) in theorem binding. Furthermore, when used as a plug-and-play parser to provide TOFL context, Thoth significantly boosts the geometry QA accuracy of other downstream MLLMs, and its visual-theorem grounding ability successfully generalizes to unseen theorems and entirely different domains, such as physics.

Critical strengths

The Core Insight is Spot-On: Current Multimodal LLMs (MLLMs) are basically glorified edge-detectors when it comes to geometry. They can see a line, a circle, and a triangle, but they don't see "the altitude of an isosceles triangle." The authors correctly identify that you cannot solve geometry without modeling theorems as first-class citizens, not just post-hoc reasoning steps.

The "Theorem-Local Reconstruction" Reward is Brilliant: This is my favorite part of the paper. Instead of just asking the model to predict coordinates or names, they force it to render the specific sub-diagram that supports a theorem. If the model claims the "Pythagorean theorem" applies to a triangle, the reconstruction will fail if the triangle isn't actually a right triangle. This is a highly creative, self-supervised way to enforce visual grounding.

Excellent RL Formulation: The three reward functions (binding, reconstruction, constraints) perfectly align with the three things you need for mathematical reasoning: identifying the rule, verifying it visually, and ensuring it makes sense in the context of the problem.

Plug-and-Play Utility: I really appreciate that Thoth isn't just an end-to-end VQA model. Framing it as a parser that outputs TOFL, which can then be fed to any frozen reasoner (even GPT-5.6 or Gemini), is a highly practical contribution. It proves that improving the visual representation helps the reasoning, independent of the LLM's raw intelligence.

Critical weaknesses

The "Single Path" Problem: Geometry is beautiful because there are often 5 different ways to prove a theorem. The paper trains Thoth to select "question-relevant theorems" based on a single reference solution. If a student (or Thoth) finds a perfectly valid, alternative proof path, the RL reward will actually penalize them. This is a known issue in theorem-proving datasets, but it feels particularly restrictive here.

The Marginal Gains for Frontier Models: In Table 3, giving TOFL to GPT-5.6 or Gemini only improves their accuracy by ~1%. If the goal is to help the best models in the world, a 1% gain might be too small to justify the computational overhead of running a separate 7B parameter parser. The gains are huge for smaller models, but the frontier gains are modest.

The "Closed-World" of 90 Theorems: I understand that 90 theorems cover a lot of high school geometry, but the real world (and Olympiad math) is much more open-ended. The authors show some zero-shot generalization to unseen theorems, which is great, but a practical geometry engine would need a way to dynamically retrieve or define new theorems.

Questions for authors

First, regarding the downstream QA results in Table 3, I noticed that the improvements for frontier models are quite small, such as a +0.93 gain for GPT-5.6 on GeoQA. Could you clarify if these gains are statistically significant across multiple runs? If the improvements for the strongest models fall within the margin of noise, this would significantly weaken the claim that Thoth acts as a universally effective plug-and-play parser for state-of-the-art reasoners, rather than just a helpful shortcut for weaker models.

Second, I have a concern about the RL reward design for theorem selection. Because the "question-relevant" theorems are extracted from a single reference solution, the Tversky reward (Eq. 19) likely penalizes the model for choosing mathematically valid alternative theorems that simply weren't used in that specific solution path. How frequently does this happen, and do you worry this forces the model to mimic the dataset's specific quirks rather than learning true, multi-path deductive reasoning?

Third, regarding the baseline comparisons in Tables 1 and 2, I understand from the appendix that general MLLMs were provided with the TOFL format prompt. However, generating such a complex custom syntax zero-shot is notoriously difficult for LLMs. Did the baseline models suffer from high rates of syntax or parsing failures? If their low scores are largely due to formatting struggles rather than a failure to perceive the geometry, the massive performance gaps reported in the tables may be artificially exaggerated.

As for future directions, since you acknowledge that the model is currently limited to a predefined library of 90 theorem templates, a highly valuable next step would be integrating a Retrieval-Augmented Generation (RAG) mechanism. This could allow the model to dynamically fetch relevant theorems and their visual patterns from an external database during inference, moving beyond a closed-world assumption. Finally, extending this framework to 3D solid geometry, as you briefly mention for ThothV2, would be a fantastic test of its generalizability. Explicitly binding 3D theorems to their 2D projections would be a rigorous way to prove whether the visual-theorem grounding paradigm truly scales beyond planar diagrams.

5 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/4)

We genuinely thank the Reviewer for the careful reading and very constructive feedback. Please find below our explanations addressing your concerns.


W.1 / Q.2: How do multiple solution paths for a math problem affect the model?

Thank you for this insightful comment. Indeed, a mathematical problem may have multiple valid solution paths involving different theorems.

We understand that your concern is that this multiplicity may lead to unnecessary penalties during training. When similar problems have inconsistent solution paths in the dataset, the model may receive conflicting GT supervision, even though all labeled theorems are valid. In our view, this is a potential data annotation issue arising from the inherent diversity of mathematical reasoning, and enumerating all possible solutions is extremely difficult and sometimes even infeasible.

However, we would like to clarify that this issue is very limited in our collected dataset, due to the high consistency of the solution paths for similar problems. Since our training dataset contains 4.5K samples, manually examining all of them in a short time is impractical. Therefore, we used GPT-6.1 Sol high to examine our dataset for consistency.

Specifically, we provided GPT-6.1 Sol high with the problem diagrams, problem texts, and our training labels:

  • We asked it to first identify groups of "repeated problems" that differ in variable names, numerical values, or diagram layouts.
  • We then asked it to count the number of samples in these groups, compute relevant statistics, and examine whether similar problems share the same solution paths (i.e., theorem labels).
  • We also manually spot-checked the results to ensure their accuracy.

Based on this analysis, we identified the following statistics on similar problems in our dataset:

Problem Type# Samples% of Total (4.5K)# Groups# Groups with Identical SolutionConsistency Rate
Proof105523.44%32531998.15%
Calculation1092.42%373491.89%
Total116425.87%36235397.51%

We define a consistent group as one in which all problems share the same reference solution (e.g., applying the Pythagorean theorem to find the length of a side in a right triangle).

The table shows that:

  • Among the 4.5K samples in our training set, only a small portion of 25.87% consists of repeated problems that differ only in variable names, numerical values, or diagram layouts, while their problem types are essentially the same.
  • These samples cover 362 problem types, among which 97.51% of the groups have identical reference solution.
  • This indicates that similar problems are relatively rare in our dataset, and cases where similar problems have inconsistent theorem labels (2.49%) are even rarer.
  • According to our statistics, this accounts for 206/4500 = 4.58% of the training data. The model therefore encounters inconsistent supervision only very infrequently during training.

Therefore, the impact of theorem multiplicity on model learning is limited, as the model receives consistent supervision signals to predict the same solution path for similar problems.

Nevertheless, we acknowledge that this may lead the model to learn dataset-specific reasoning patterns. However, this does not currently affect its ability to help solving downstream mathematical problems, as solving a problem only requires one correct reasoning path rather than enumerating all possible paths.

We have added discussion on this issue in Appendix C.3. In our future work, we will study enabling the model to learn diverse reasoning paths, and deal with path inconsistency through noise label modeling -- such a standard approach can hadle contradictory labeling scenarios.

Thank you again for your very thoughtful suggestions.

AuthorsOpenReview ↗

Official Comment by Authors (2/4)

W.2 / Q.1: Are the improvements for frontier models statistically significant?

We sincerely thank you for raising this question. We would like to clarify the following points:

  • First, frontier models are not the primary beneficiaries of Thoth. TOFL explicitly represents theorem-related information in geometric diagrams, reducing visual perception and theorem selection errors, particularly for smaller models. As you pointed out, the gains on open-source models, which require no paid API calls, are substantial. For example, Qwen3-VL-32B achieves improvements of +6.90 on GeoQA and +6.78 on MathVerse, making Thoth particularly attractive for cost-effective models/applications.

  • Second, we emphasize that the improvements are not uniformly modest across frontier models. For example, Gemini-3.7 Flash achieves gains of +1.32, +0.80, +1.16, and +2.54 percentage points across the four benchmarks, respectively, which we consider relatively large improvements.

  • Third, although the improvements on GPT-5.6 Sol are relatively modest on some benchmarks, they are statistically significant. To verify this, we conduct three independent runs for both GPT-5.6 Sol and Gemini-3.7 Flash on two larger benchmarks, PGPS9K with 1000 samples and MathVerse with 3940 samples, to reduce the influence of random variation. Across runs, we use semantically equivalent prompt paraphrases while keeping all other input information, such as TOFL representations and problem statements, unchanged. We then perform t-tests on the resulting accuracies to assess statistical significance.

ModelDatasetVanilla Run 1Vanilla Run 2Vanilla Run 3+TOFL Run 1+TOFL Run 2+TOFL Run 3Improvementp-value
GPT-5.6 SolPGPS9K82.7082.6082.8082.9082.9083.00+0.230.019804
GPT-5.6 SolMathVerse80.9180.4379.6281.8881.8881.45+1.410.029840
Gemini-3.7 FlashPGPS9K89.7089.8089.7090.5091.0090.30+0.870.039012
Gemini-3.7 FlashMathVerse88.5388.5387.2391.0791.5290.99+3.100.012914

These results suggest that, despite the modest gains in some cases, the improvements remain statistically significant, with p-values below 0.05.

While frontier models possess strong visual understanding and reasoning capabilities (albeit these are closed models and mechanisms/their scale are not known), there is a substantial cost associated with their use.

Regarding efficiency:

Running an additional parser introduces a small computational overhead. However, we believe the key benefit of Thoth for closed-source models is its ability to reduce the time and reasoning tokens consumed by chain-of-thought reasoning, particularly when reasoning about relevant theorems, thereby reducing API costs. To demonstrate this, we conducted an experiment using GPT-5.6 Sol on 100 randomly sampled MathVerse QA examples.

SettingQA Acc.Total Input TokensTotal Output TokensReasoning TokensTotal TokensAvg. Time per SampleTotal Cost
Reasoning=None79%68,2701,100069,3702.03 s$0.31
Reasoning=None + TOFL82%130,6801,0900131,7702.68 s$0.33
Reasoning=High83%70,07028,96020,51099,0305.89 s$0.87

As shown above, Reasoning=None + TOFL achieves comparable accuracy to Reasoning=High, while Reasoning=High incurs 2.64× the API cost and 2.20× the inference time. Since Thoth can be deployed locally as an open-source plug-and-play parser, its relatively small local computational overhead (single A100 local GPU running for ~1s) offers a promising way to reduce API costs while achieving similar performance. With our model, it is also easier to understand how it performs recognition.


Furthermore, as discussed in Lines 1022–1024, Thoth only needs to run once per diagram with negligible cost. The resulting TOFL representations can be cached and shared across different downstream reasoners. This allows users to select reasoners with different capabilities and pricing according to the needs, without rerunning Thoth.

We sincerely thank you again for raising this important question and hope that our additional analysis fully addresses your concern.

AuthorsOpenReview ↗

Official Comment by Authors (3/4)

Q.3 Are the low baseline scores caused by formatting errors?

Thank you for this question. The low scores of general-purpose MLLMs in Tables 1 and 2 are not caused by formatting errors, but rather reflect their limitations in geometric perception.

We use carefully designed formatting prompts (as briefly illustrated in Figure 9) to guide MLLMs toward the required output structure, and iteratively refine these prompts through manual inspection of model outputs to ensure their effectiveness and reliability. During inspection, we observed occasional formatting issues in certain models (e.g., concatenating multiple TOFL statements into a single line). We address such issues through minimal post-processing that preserves the original semantic content of the predictions.

To further investigate whether formatting errors account for the low baseline scores, we report the F1 scores for individual geometric primitives alongside the format validity rates:

ModelPoint F1Segment F1Line F1Circle F1Format ↑
GPT-5.6 Sol0.54510.23920.11750.170197.17%
Gemini 3.7 Flash0.54840.03680.00330.131698.00%
Qwen2.5-VL-7B0.05680.06370.01250.032088.00%
Qwen3-VL-8B0.08920.06990.04030.004492.83%
LLaVA-OneVision-2-8B0.06070.12530.01250.081394.67%
InternVL3.5-8B0.04830.03540.02050.016789.50%
Gemma-4-12B0.08220.06740.08250.006592.50%
Thoth0.95820.72090.61270.902098.67%

Format validity is measured by whether the predicted object declarations can be successfully parsed according to the required syntax. As shown in the above table, most general-purpose MLLMs achieve relatively high format validity rates, indicating that formatting errors alone cannot explain the substantial performance gap between these models and Thoth.

This also reveals that the dominant errors arise from inaccurate geometric perception, such as large deviations in predicted point coordinates, incorrect segment endpoints, and inaccurate circle centers or radii, which persist even when the outputs are correctly formatted.

Reviewer_AiMbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Thoth proposes a theorem-oriented method for analyzing plane geometric figures. While identifying geometric elements such as points, lines, and circles, it predicts theorems applicable to the figure and the current problem, and explicitly binds them to the corresponding objects, providing a structured basis for subsequent problem-solving. Key contributions include the formal language TOFL, the GT-210K dataset comprising 210,000 samples covering 90 theorem templates, and a training framework that combines supervised fine-tuning with reinforcement learning to optimize analysis quality through theorem binding, figure reconstruction, and problem constraint optimization. In terms of validation, the authors evaluated element parsing and theorem recognition on both synthetic and real-world geometric figures and tested downstream performance on four geometry question-answering benchmarks; the results show improved problem-solving accuracy for most model-task combinations, and component ablation studies further support the role of task-relevant theorem selection and explicit binding.

Critical strengths

The paper unifies the modeling of geometric elements, theorem instances, and their visual grounds, and distinguishes between theorems applicable to a given scene and those relevant to a specific problem, thereby providing a clear and interpretable intermediate representation that bridges graphic perception and mathematical reasoning. Its validation covers analytical quality, theorem identification, and downstream problem-solving, while component ablation demonstrates the independent value of theorem selection and object binding;

Critical weaknesses

The consistency of synthetic data labels requires further clarification: the appendix marks the entire theorem block for difficult negative examples as “None,” but the fact that a key premise of a particular theorem is invalidated does not necessarily rule out the applicability of other relevant theorems; the paper does not explicitly state whether all theorems were re-verified for these negative examples. Second, although component-level ablation studies have been conducted, the main comparison still focuses primarily on the general-purpose MLLM, lacking a direct comparison with existing specialized geometric analysis methods under comparable data and training budgets.

Questions for authors

In the theorem block for difficult negative examples, the label “None” is used—does this mean that the target theorem in question is no longer valid, or that none of the 90 theorem categories have applicable instances? After generating negative examples, are all theorems re-examined, and are instances that remain valid retained? This directly affects the reliability of training labels and theorem recognition evaluations. Second, for the three variants in Table 4—“primitives only,” “theorem names only,” and “full TOFL”—were the same data split, training phase, number of optimization steps, and real QA samples used?

4 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/3)

We sincerely thank the Reviewer for taking the time to carefully read our paper and provide such valuable feedback. Please find our detailed answers below.


W.1 / Q.1: Clarification on the use of "None" in theorem labels for hard negative samples

Thank you for raising this important question.

We apologize that the original manuscript lacked sufficient detail and clarify our labeling strategy below.

As described in Appendix A.2, our negative samples include 5% theorem-free negatives and 15% hard-premise negatives:

  • For the 5% theorem-free negatives, we use None to indicate that no theorem from the 90 categories applies to the scene. These are typically very simple scenes, and our generator explicitly ensures that no theorem is applicable.

  • For the 15% hard-premise negatives, we clarify that only theorems with violated premises are removed from the labels. These examples violate only one decisive premise of a target theorem, without affecting any other premises (that are kept). Note that the theorem closure of each scene is computed after this violation (Lines 737–738), which identifies all applicable theorems based on the geometry in the scene. Therefore, violating one theorem's premise may also invalidate other theorems. For example, removing the right-angle condition of the Pythagorean Theorem also invalidates the Right Triangle Median to Hypotenuse Theorem. We assign None to an empty section only when no valid theorem remains, explicitly enabling the model to learn to predict when no theorem is applicable.

We further report the distribution of the number of theorems affected by premise violations per sample among the hard-premise negatives in our proposed GT-210K dataset:

# Affected Theorems123≥4 (including None)Of which: None
Percentage51.81%36.16%9.43%2.60%1.20%

All generation rules are programmatically implemented in our data engine, which will be open-sourced upon acceptance. During development, we ensured that premise violations do not affect unrelated theorems. The generated samples also undergo both automated verification and manual inspection to ensure correctness.

We have revised the manuscript to clarify this point. Thank you again for encouraging us to clarify this point.

AuthorsOpenReview ↗

Official Comment by Authors (2/3)

W.2 Comparison with existing geometric analysis methods

Thank you for this constructive suggestion.

We further evaluate a series of models specifically fine-tuned for mathematical tasks, including:

  • G-LLaVA-7B (Gao et al., 2023),
  • Math-LLaVA-13B (Shi et al., 2024),
  • URSA-8B (Luo et al., 2025),
  • SVE-Math-Qwen2.5-7B (Zhang et al., 2025),
  • GDP-4B (Wang et al., 2026),

on both visual perception and theorem selection tasks, following the same experimental settings as Tables 1 and 2 in the main paper.

Please note that these available open-source mathematical models are generally designed specifically for reasoning tasks, i.e., they are tuned to take geometry diagrams and problem statements as input and generate final answers or intermediate solution steps. Such task-specific methods often exhibit weaker general instruction-following capabilities compared to general-purpose MLLMs.

Consequently, even when provided with detailed prompts specifying the TOFL format, they frequently produce incorrectly formatted parsing outputs or even corrupted text:

  • This makes it difficult to fairly evaluate their primitive perception and theorem selection capabilities through TOFL generation and diagram reconstruction.
  • This is also why we did not initially include math-specific methods in our parsing comparisons.

Nevertheless, we provide their results under the same evaluation settings as Thoth for reference and further analysis.

Primitve Perception:

ModelSyntheticReal
MSE ↓SSIM ↑DINO ↑MSE ↓SSIM ↑DINO ↑
GDP-4B0.98020.01920.00670.95170.04310.0051
URSA-8B0.02060.86570.52290.04760.80150.5308
G-LLaVA-7B0.23250.73050.28950.60770.33580.0858
SVE-Qwen2.50.02890.81880.44320.03190.79340.4957
Math-LLaVA0.01870.83890.42650.03720.82670.4176
Thoth–RL0.00680.97080.97500.03450.81390.7982

Theorem Recognition:

ModelScene Thm. F1 ↑Selected Thm. F1 ↑
GDP-4B0.10830.0165
URSA-8B0.17970.4126
G-LLaVA-7B0.12250.3824
SVE-Qwen2.50.04400.0781
Math-LLaVA0.09370.0128
Thoth–RL0.85910.8634

Since their outputs often deviate from the format and theorem names defined in TOFL, we use GPT-6.1 Sol to standardize their outputs and assess their semantic consistency with the ground-truth annotations, ensuring a relatively fair comparison.

The comparison may be more appropriate when evaluating these methods on the QA tasks they are specifically tuned for:

ModelSettingGeoQAGeometry3KPGPS9KMathVerse
GDP-4B-2.920.170.000.51
URSA-8B-70.95*39.2739.8039.29
G-LLaVA-7B-54.91*23.00*26.8020.46
SVE-Qwen2.5-75.99*14.98*41.4033.50
Math-LLaVA-45.49*8.15*30.3020.23
Ours (Qwen3-VL-8B)Vanilla58.2238.9443.4027.69
+TOFL60.61 (+2.39)42.93 (+3.99)46.90 (+3.50)31.73 (+4.04)
Ours (Gemini-3.7-Flash)Vanilla97.3595.5189.7088.53
+TOFL98.67 (+1.32)96.67 (+1.16)90.50 (+0.80)91.07 (+2.54)

* indicates that the model was trained on the corresponding benchmark's training data or its derivatives.

AuthorsOpenReview ↗

Official Comment by Authors (3/3)

As shown in the results:

  • Combining Thoth's outputs with different downstream reasoners outperforms most previously proposed math-specialized methods.
  • Notably, even an 8B model such as Qwen3-VL significantly surpasses the 13B Math-LLaVA.
  • These results suggest that general-purpose MLLMs already possess strong capabilities for solving geometry problems, while their main limitations lie in accurate primitive perception and theorem selection.
  • Our approach specifically focuses on improving diagram parsing and provides the resulting structured representations to strengthen downstream reasoning.

More importantly, as MLLM architectures continue to evolve, our framework allows us to readily replace the downstream reasoner with a newer and stronger model to achieve better performance. Our method continues to provide additional improvements even when paired with highly capable reasoners (e.g., Gemini-3.7-Flash). In contrast, math-specialized models cannot directly benefit from improvements in their underlying base models and require substantial computational resources for retraining to keep up with these advances.

We also provide a comparison of the training data budgets for reference:

ModelModel SizeTotal DataReal DataSynthetic DataAnnotation Type
G-LLaVA7B177.5K8.1K169.4KCaption, QA
Math-LLaVA13B360K40K320KQA
URSA8B1.86M860K1MCaption, CoT
SVE-Math7B400.1K49.4K350.7KBoxes, Junctions, QA
GDP-4B4B25.9K25.9K0Code representation
Thoth7B209.5K9.5K200KTOFL, Theorem Names

G-LLaVA has the closest training budget to ours. Nevertheless, even on the QA tasks for which G-LLaVA is specifically trained, our method, when combined with more recent MLLMs, achieves higher accuracy. This further demonstrates a key advantage of our approach: it can readily benefit from advances in MLLMs by transferring to stronger backbone models as they evolve. Notably, our method also uses less real data than most of the compared methods, as it does not directly learn to solve QA tasks but instead improves downstream reasoning indirectly through precise theorem-oriented diagram parsing.

Thank you again for the Reviewer's suggestion. We would also love to know if there are any other mathematical MLLMs that could be applied to these tasks, and we'd be happy to run them for you.

Q.2 TOFL ablation configuration consistency in Table 4

Yes, we strictly controlled the training configurations across all three ablation variants to ensure a fair comparison. Specifically, all variants share the same data split, SFT training protocol, number of optimization steps, and hyperparameters, differing only in the construction of supervised TOFL representation. We report the key training configurations in the following table:

Configuration+Primitive-only+Theorem name-onlyFull TOFL
Data split on GT-210K (Train/Val/Test)200K / 9.4K / 600200K / 9.4K / 600200K / 9.4K / 600
Training methodFull-parameter SFTFull-parameter SFTFull-parameter SFT
Training duration2 epochs2 epochs2 epochs
Global batch size256256256
Learning rate (Vision / Projector & LLM)2e-7 / 1e-62e-7 / 1e-62e-7 / 1e-6
Distributed trainingFSDP, 16 NVIDIA V100 GPUsFSDP, 16 NVIDIA V100 GPUsFSDP, 16 NVIDIA V100 GPUs
TOFL designPrimitives onlyPrimitives + theorem names (w/o binding info)Full TOFL

We would also like to highlight that all training configurations are consistent with those used in the main experiments, as detailed in Appendix B.

However, we would like to clarify that no real QA samples were used in this diagnostic experiment, as raised in your question. Our goal is to leverage the complete annotations available in synthetic data to isolate the contributions of different TOFL blocks and investigate how primitive understanding and theorem grounding mutually reinforce each other. Therefore, all three variants were trained exclusively on GT-210K and evaluated on the same synthetic test split. For further evidence of how the capabilities learned from synthetic data generalize to real QA diagrams, please refer to our response to Reviewer pd6o, W.1.


We would like to thank again the Reviewer for the constructive feedback. We hope that our responses have addressed all the concerns.

Esteemed Reviewer,

As the author-reviewer discussion deadline approaches, we would like to kindly follow up on our responses to your review.

If anything remains unclear, kindly let us know. We would be happy to answer any further questions and make additional revisions if needed.

We would be grateful if you could consider updating your rating or letting us know if our responses have satisfactorily addressed your concerns.

Best regards,

Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Authors’ Responses

Dear PC, SAC, AC, and Reviewers,

We sincerely thank for Reviewers' constructive feedback and expert insights, which have helped us further clarify our contributions and strengthen our manuscript.

We are particularly encouraged by the Reviewers' recognition of Thoth as a well-motivated, innovative, and comprehensively evaluated work.

For your convenience, we briefly summarize below the key concerns raised and the clarifications and experimental evidence provided in our responses.

Summary of Key Reviewer Concerns and Responses

ReviewerKey ConcernHow It Was AddressedResolved
AiMbThe use of “None” labels for hard-negative samples is unclearWe clarified that valid theorems are retained in the labels. None is assigned only when no valid theorems remain.Yes
AiMbComparison with existing geometric analysis methodsWe compared five specialized mathematical baselines, covering parsing, theorem recognition, and QA, alongside training budgets.Yes
5z2kMultiple solution paths may affect model trainingWe found consistent labels in 97.51% of similar-problem groups, suggesting that the impact on training is minimal.Yes
5z2kStatistical significance of frontier model improvementsThree-run evaluations on two large benchmarks show statistically significant (p < 0.05) gains in all tested settings.Yes
5z2kWhether low baseline scores arise from formatting errorsWe reported format-validity rates and detailed primitive-level scores, demonstrating that the low score stems from inaccurate geometric perception rather than formatting errors.Yes
pd6oTheorem binding on real QA diagramsEvaluation on 100 real diagrams shows 89.49% binding accuracy and 76.29% joint correctness, indicating that the binding capability learned from synthetic data transfers well to real-world diagrams.Yes
pd6oMechanism of theorem-local reconstruction rewardWe clarified its indirect contribution to binding performance through improved local geometric understanding. Separate ablations show that the local reward alone improves binding accuracy by 2.60 percentage points.Yes

We have made the following revisions to the paper, highlighted in blue:

RevisionPage / Section
Clarified the role of theorem-local reconstruction in improving theorem–primitive binding.P. 5, Sec. 3.2
Clarified the hard-negative design and "None" label assignment.P. 16, App. A.2
Added ablation experiments on theorem-local reconstruction rewards.P. 23–24, App. C.2
Added a discussion on solution multiplicity and an analysis of label consistency in real QA training data.P. 24, App. C.3
Made minor formatting adjustments.P. 4, P. 8, Fig. 4(c)

We hope that our responses have satisfactorily addressed the concerns raised by the Reviewers. We sincerely thank you once again for your time, careful consideration, and valuable guidance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.