acceptodds
Under review as a conference paper at ICLR 2027

TeXForm: Auditing and Improving LaTeX Formula Normalization in Data and Evaluation Pipelines

Abstract

LaTeX normalization maps the many equivalent forms of a formula to one canonical string. Formula-recognition data pipelines and the formula tracks that rank today's vision-language models all silently rely on normalizers, yet not one of these normalizers states what it preserves. Our experiments reveal failures accumulated invisibly for years: normalization in dataset construction changes what its inputs render to, with case studies exposing symbol loss and structural truncation; inside three widely used benchmarks (UniMER-Test, OCRBench v2, and OmniDocBench), the official scoring pipelines falsely reject 19–26% of render-equivalent prediction–label pairs, with subset rates spanning 6–55%, while accepting some pairs that render differently as false-accept candidates. Beyond traditional text metrics, CDM, the recently proposed render-aware metric, also embeds a normalizer that alters what 14% of inputs render to. To address this systematically, we introduce LaTeX normalization as a formal task with four measurable properties (safety, canonicity, idempotence, and coverage), and characterize when normalize-then-match is a valid equivalence test. Guided by this formalization, we build TeXForm, a LaTeX normalizer driven by an explicit knowledge base of command argument structure and rewrite rules. Among all normalizers we evaluate, TeXForm is the only one to combine substantial canonicity (37–47%) with render-safety above 97%, the property that normalized training labels require. On UniMER-Test and OmniDocBench, replacing built-in normalization with TeXForm recovers 38.05% and 89.83% of previously rejected render-equivalent pairs, respectively. The audited normalization configurations change model rankings, including a leader reversal on UniMER-Test. We release TeXForm, including its knowledge base, as open source.

Primary area:
datasets and benchmarks
Keywords:
LaTeX normalization, mathematical expression recognition, document parsing, OCR, benchmark evaluation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_99JzOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

This paper proposes TeXForm, a LaTeX normalization method based on a command knowledge base and syntax-tree rewriting. It aims to preserve formula content while unifying equivalent representations, substantially reducing false rejects of render-equivalent formula pairs on selected benchmarks.

Critical strengths
  • Thorough evaluation across multiple datasets and benchmarks, including ablations, cross-renderer validation, and sensitivity analyses.
  • Strong normalization results, combining high rendering safety with substantial canonicalization and reducing false rejects on UniMER-Test and OmniDocBench.
Critical weaknesses
  • TeXForm follows the established parse–rewrite–serialize pipeline. Its engineering contribution is substantial, but its methodological novelty is limited.
  • Safety and canonicity are reported separately, so it is unclear how many successfully normalized pairs also retain their original rendering.
Questions for authors
  • Could the authors report, among render-equivalent formula pairs normalized to the same LaTeX string, how often the shared output renders differently from the original formulas? This would help clarify how much of the merging preserves the original rendering.
3 comments
AuthorsOpenReview ↗

Response to Reviewer 99Jz (1/2): Novelty and Joint Safety–Canonicity Audit

Thank you for your thorough review, for highlighting the gap between the separate safety and canonicity evaluations in Table 1, and for your candid feedback on presentation.

Following your suggestion to evaluate safety and canonicity jointly, we added a safe pair canonicity column to Table 1. It measures the share of equivalent pairs folded without changing the rendering. We placed it in the main table because it directly tests both conditions of Proposition 1 (Section 4).


Weakness 1: Novelty Beyond the Parse-Rewrite-Serialize Pipeline

We agree that the parse-rewrite-serialize pipeline itself is standard. Our contribution is to turn an ad-hoc preprocessing step into a formally specified, auditable component of the ML pipeline. In fact, existing normalizers (im2markup, MathNet, CDM) do not even rewrite a well-formed AST, instead pairing a lossy KaTeX pass with structure-blind regex or token substitutions (Section 3). We go beyond that pipeline in three ways:

  • A checkable definition of correct normalization, where prior normalizers state no guarantee. We formalize LaTeX normalization into four measurable rendering-based properties and state when normalized matching is valid (Proposition 1), so that both normalizers and string-based metrics can be systematically tested for the first time (Section 4).
  • Evidence that this unaudited step corrupts both data and evaluation, independent of TeXForm. Using this definition, our audit shows that inherited normalizer defects silently corrupt training labels and distort benchmark verdicts, even inside render-aware metrics such as CDM (Sections 3, 6).
  • Normalization driven by knowledge derived from the renderer. Instead of regexes and token tables, TeXForm derives a human-adjudicated knowledge base of command specifications offline from MathJax's source and enforces preservation contracts on rewrite rules, making deterministic AST normalization practical across informally documented macro packages (Section 5).

The repaired baselines built for Reviewer WZKC isolate this third point. They keep the same parse-serialize skeleton and fix all nine documented defects in im2markup and MathNet without adding any command knowledge, and they reach at most 17.45% safe pair canonicity, against 37.15% for TeXForm (Faithful) (new Appendices C.7–C.8). Since both sides share the skeleton, the gap comes from the knowledge base and contracts.


Weakness 2: Joint Evaluation of Safety and Canonicity

For TeXForm (Faithful), only 0.91% of folded pairs render differently from the original formulas (1.32% for Corpus and 0.81% for Authoring). By contrast, MathNet changes the rendering of 75.09% of the pairs it folds, and CDM changes 20–22% of theirs.

Your concern points to a real gap in how we originally reported the two properties. Proposition 1 makes normalized matching a valid test of rendering equivalence only when safety and canonicity hold on the same inputs, yet Table 1 measured safety on raw labels and canonicity on model-prediction pairs. Measured apart, the two rates cannot separate folding that comes from knowing equivalent forms from folding that changes the rendering. For example, a normalizer that maps every formula to one fixed string folds every pair (100% pair canonicity) while preserving no rendering.

Following your suggestion, we ran a new audit across all 29,528 equivalent pairs (new Appendix C.8, Table 15). For each folded pair (x,y)(x, y) with shared output zz, we check z≡xz \equiv x and z≡yz \equiv y, which is exactly safety on both members, so safe pair canonicity counts the pairs on which both premises of Proposition 1 hold at once:

NormalizerPair canonicity (%)Changed rendering among folded pairs (%) ↓Safe pair canonicity (%) ↑
im2markup18.651.4718.37
MathNet66.4275.0916.53
CDM (three releases)24.96–25.1420.31–22.3419.38–19.93
TeXForm (Faithful)37.500.9137.15
TeXForm (Corpus)46.581.3245.96

Once rendering-altering folds are excluded, no prior normalizer reaches 20% safe pair canonicity, which resolves the apparent trade-off with MathNet, since three quarters of its folds change the rendering. Table 1 now reports safe pair canonicity next to pair canonicity, and Appendix C.8 gives the full 12-row comparison.

Even in the ~1% of folded pairs with a pixel difference, TeXForm usually only shifts glyph positions while preserving MathML structure (e.g., (5^{\prime\prime}) vs. ( 5'' )).

This joint criterion also shows the first point of Weakness 1 at work. Because the four properties are defined formally, the criterion follows directly from Proposition 1 instead of being chosen ad hoc, and it turns the apparent safety–canonicity trade-off among prior tools (Figure 3) into a measurable gap between pattern-based normalizers and TeXForm.

AuthorsOpenReview ↗

Response to Reviewer 99Jz (2/2): Direct Answer to the Question and Presentation Revisions

Question 1: Among render-equivalent pairs normalized to the same string, how often does the shared output render differently from the originals?

Rarely for TeXForm: only 0.91% of the pairs TeXForm (Faithful) folds (1.32% for Corpus), against 75.09% for MathNet and 20–22% for CDM. See Weakness 2 above for the full audit over all 29,528 equivalent pairs and the new safe pair canonicity column in Table 1.


Presentation (Score 1): Revisions to Improve Clarity

Taking your presentation feedback seriously, we used the extra discussion-period page to expand the main text and spell out parts that had been overly compressed (all changes marked in blue; full list in our general comment). We reworked the presentation of Tables 1 and 2 in Sections 6.2 and 6.4. In Section 5, we added a dedicated Normalization profiles paragraph and expanded the Parsing and Transforming descriptions, and in Section 6.1 we added an upfront Supports and rates paragraph.


We hope these new results and manuscript revisions address your concerns. If any passage still feels dense, please let us know so we can refine it further. We would also appreciate your reconsideration in light of these revisions.

AuthorsOpenReview ↗

Follow-up to Reviewer 99Jz

Dear Reviewer 99Jz,

Thank you again for your review. With about one day left in the discussion period, we would like to check whether our response and revision resolve your concerns. Below is a brief summary of our answer to each of your points, the first two of which directly shaped revisions to the main text.

Joint evaluation of safety and canonicity. Following your question, Table 1 now has a new column, safe pair canonicity, which counts a pair only when the shared output renders like both original formulas (defined in Section 6.1, discussed in Section 6.2). It tests both conditions of Proposition 1 on the same pairs. To answer your question directly, TeXForm (Faithful) changes the rendering of only 0.91% of the pairs it normalizes to the same string, against 75.09% for MathNet (new Appendix C.8).

Presentation. We used the extra page to rewrite the parts of the main text that were too compressed. Section 5 now describes the four profiles and walks through the three stages with the example in Figure 2, Section 6.1 introduces the evaluation data and each rate before the results, and Tables 1 and 2 have a simpler layout. All changes are marked in blue.

Methodological novelty. We agree that the parse–rewrite–serialize pipeline itself is standard. Our contributions are the definition of LaTeX normalization as a task with four measurable properties (Section 4), the audit of deployed normalizers, datasets, and benchmarks that this definition makes possible (Sections 3 and 6), and the command knowledge base that drives TeXForm (Section 5). The repaired normalizers we built for Reviewer WZKC also show that this command knowledge, rather than the pipeline, accounts for most of TeXForm's gain.

We would appreciate knowing whether these changes resolve your concerns, and whether the revised sections read more clearly. If so, we would be grateful if your final assessment could reflect this.

Best regards,

The Authors

Reviewer_WZKCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies LaTeX normalization in formula-recognition data and evaluation pipelines. The authors audit existing implementations for rendering safety, canonicity, idempotence, and coverage, showing how preprocessing can alter labels, reject visually equivalent predictions, and change model rankings. They introduce TeXForm, which combines an explicit command knowledge base, syntax-tree rewriting, and stable serialization, and evaluate it on existing datasets and three benchmark tracks.

Critical strengths
  • The paper highlights an easily overlooked practical issue: LaTeX notation and preprocessing choices can affect formula-recognition scores and even model rankings. Making these problems explicit is useful for researchers who rely on these datasets and evaluation tools.

  • The experiments examine normalizers, dataset labels, and evaluation pipelines, with component ablations, cross-renderer checks, threshold sensitivity, and residual-error analyses.

Critical weaknesses
  • For work in datasets and benchmarks, I expect it to reveal important new evaluation problems or provide a validated, more trustworthy evaluation protocol. The LaTeX normalization tool has practical value, but changes in scores and rankings alone do not establish that model capabilities are assessed more accurately, so I remain unconvinced by the strength of its evaluation contribution.

  • The paper identifies several specific implementation errors in existing tools, but does not compare TeXForm with versions that correct these errors, making it difficult to assess the additional benefit beyond repairing existing tools. It also provides no runtime or memory measurements, leaving the practical cost of using TeXForm for large-scale data processing and evaluation unclear.

Questions for authors
  • Appendix C.3 mentions that Uni-MuMER uses its own normalization cleaner, which is excluded from the comparison. Could you clarify the reason for excluding it and whether it differs substantially from the evaluated MathNet implementation?

  • Could you compare TeXForm with a representative existing tool after fixing the main implementation errors identified in the paper, to clarify the additional benefit of TeXForm? Could you also report runtime or throughput and memory usage for TeXForm and existing tools using the same hardware and input data?

3 comments
AuthorsOpenReview ↗

Response to Reviewer WZKC (1/2): Evaluation Contribution and Repaired Baselines

Thank you for your careful review and constructive questions. We address each point below with new experiments added to the revised appendices.

Weakness 1: Strength of the Evaluation Contribution

We agree that changes in scores or rankings alone cannot show that models are assessed more accurately. We address the three parts of this concern in turn.

Does the paper reveal an important new evaluation problem? Yes: benchmark scoring itself misjudges model outputs. Before a formula benchmark compares a model's prediction with its reference, a normalizer rewrites both LaTeX strings to unify equivalent notations. This step has been reused for years without audit and wrongly rejects 19–26% of predictions that render identically to their references on UniMER-Test, OCRBench v2, and OmniDocBench (Table 2). The problem is not limited to string metrics: CDM, the render-aware metric designed to avoid string comparison, runs a normalizer from the same lineage that alters the rendering of ~14% of its inputs before scoring (Section 3, Table 1). These are errors in how models are evaluated, not only in how data are prepared.

Does the paper provide a validated, more trustworthy evaluation protocol? Yes: a rendering-based protocol that audits benchmark misjudgments and validates scoring pipelines. In Section 4, we define four measurable properties and state when normalized matching is valid: a normalizer must preserve rendering (safety) and map formulas that render identically to one string (canonicity; Proposition 1). Our protocol tests any benchmark against this standard: render the raw prediction and reference, compare the images, and count every benchmark decision that disagrees. We apply it to six deployed normalizers and three benchmarks, and the findings hold under KaTeX, XeTeX, and strict pixel identity (Appendix F). Replacing legacy normalizers with TeXForm turns existing metrics into a validated scoring pipeline.

Do the score and ranking changes show more accurate evaluation? Not by themselves, so we verify each changed decision directly. We keep each benchmark's scoring rule, replace only the normalizer, and check every changed decision against images rendered from the raw strings, untouched by TeXForm or any other normalizer. On UniMER-Test, replacing the normalizer recovers 12,278 predictions that render identically to their references yet were scored as wrong, while losing 7 true accepts (6 on inputs TeXForm declines to normalize). Accepted pairs that render differently also fall from 220 to 163 (Section 6.4, Appendix E.4). When recognizing formulas from images, a prediction that renders identically to its reference is correct. The ranking reversal is a consequence of these corrections. We support the corrections with the image check above, not with the reversal itself.

TeXForm does not eliminate the problem: scoring with TeXForm still rejects 16.33% of render-equivalent pairs on UniMER-Test (Table 2), and the same protocol measures the remaining gap for future normalizers.


Weakness 2 (Part 1): Comparison with Repaired Legacy Normalizers

Repairing the legacy normalizers' defects closes little of the gap: TeXForm's additional benefit comes from command knowledge, not from bug fixes. We repaired all nine implementation defects that our audit identified in im2markup and MathNet (in parsing, string substitution, serialization, and exception handling), added no command specifications or alias tables, and evaluated the repaired tools under the same audit (new Appendices C.7–C.8, Tables 14–15). Safety rises by only ~1 point, neither repaired tool folds more pairs than its original, and safe pair canonicity (the share of equivalent pairs folded into an output that renders like both inputs) stays below 18%:

NormalizerSafety (%) ↑Pair canonicity (%)Safe pair canonicity (%) ↑Render drifts ↓
im2markup (original →\to repaired)94.43 →\to 95.7118.65 →\to 14.6418.37 →\to 14.6417,846 →\to 13,492
MathNet (original →\to repaired)59.65 →\to 60.8866.42 →\to 58.6716.53 →\to 17.45135,326 →\to 131,055
TeXForm (Faithful)98.8437.5037.153,264
TeXForm (Corpus)97.2046.5845.968,933

The repairs make both tools safer but fold fewer pairs. Part of their original folding came from the defects themselves, which erase distinctions between commands. Most render drifts (outputs that render, but differently from their inputs) persist because the damage is spread across uncatalogued token, font, and array rewrites rather than isolated bugs.

Folding more pairs safely requires knowing each command's argument structure and equivalent forms, which TeXForm's knowledge base supplies. Against the repaired im2markup, TeXForm (Faithful) has about 4× fewer render drifts (3,264 vs. 13,492) and folds 2.5× as many pairs while preserving rendering (37.15% vs. 14.64%).

AuthorsOpenReview ↗

Response to Reviewer WZKC (2/2): Cost, Uni-MuMER's Cleaner, and Direct Answers to Questions

Weakness 2 (Part 2): Throughput and Peak Memory on Identical Hardware and Inputs

The cost is small: on one core, TeXForm normalizes one million formulas in about 2.6 minutes using 36 MiB of memory, and 16 processes take about 12 seconds. We measured all tools on the same 23,757 UniMER-Test reference formulas and the same AMD Ryzen 9 9950X (new Appendix G, Table 37). Across 16 processes, TeXForm is the fastest of all normalizers that parse LaTeX, and in both settings it uses the least memory of all tools:

Normalizer1-Core (formulas/s)1-Core Mem (MiB)16-Process (formulas/s)16-Process Mem (MiB)
UniMERNet postprocess (whitespace-only)22,83242.1298,391412.5
im2markup*9,804165.565,3761,365.3
MathNet2,173135.023,850932.5
CDM (in UniMERNet, as released)47114.1--
CDM (in UniMERNet)*4,657160.740,0641,874.1
CDM (in OmniDocBench v1.5)*7,011153.648,6071,857.2
CDM (in OmniDocBench v1.6)818187.511,1462,682.2
TeXForm (Faithful)6,39036.283,774257.8
TeXForm (Corpus)6,33436.281,148257.3

UniMERNet postprocess only cleans whitespace with regular expressions and does not parse LaTeX, so it serves as a minimal-work speed reference. * marks baselines that we accelerated because their released wrappers were too slow for million-formula auditing. As released, CDM spawns a new process per formula and runs at only 47 formulas/s (~100×100\times slower), so we batch the JavaScript stages of CDM and im2markup inside a persistent process without changing their outputs.

Even against these accelerated baselines, TeXForm finishes all of UniMER-Test in 3.7–3.8 s on one core and <0.3 s across 16 processes (12.8–13.1×12.8\text{–}13.1\times speedup) at a fraction of the memory footprint.


Question 1: Why was Uni-MuMER's cleaner excluded, and does it differ substantially from MathNet?

We initially omitted Uni-MuMER's cleaner as an apparent variant of MathNet, but yes, it differs substantially and is now part of our audit. It descends from im2markup preprocessing and shares MathNet's JavaScript stage, modifying only vocabulary mappings and sample filtering, so we had not expected its behavior to diverge much. Taking your comment seriously, we added Uni-MuMER's cleaner to our benchmark suite and evaluated it under the same audit protocol (new in Appendices C.2, C.3, and C.8; Tables 12 and 15):

NormalizerSafety (%) ↑Coverage (%) ↑Pair canonicity (%)Safe pair canonicity (%) ↑
UniMERNet postprocess (whitespace-only)99.8399.408.368.31
MathNet59.6598.3266.4216.53
Uni-MuMER cleaner (new)70.4191.7452.1313.47
TeXForm (Faithful)98.8499.1637.5037.15
TeXForm (Corpus)97.2099.1646.5845.96

Compared with MathNet, Uni-MuMER reaches higher safety but lower coverage and even lower safe pair canonicity, as 74.15% of the pairs it folds still alter rendering. This differs from our initial expectation mainly because it invokes the built-in tokenize mode of the inherited im2markup script rather than normalize mode (bypassing im2markup's broken tree serializer) while discarding 8.3% of inputs containing common commands such as \text, \rm, or \varepsilon.

Because it still alters nearly three in ten retained formulas and drops valid inputs, it remains unsuitable as a general-purpose normalizer. Including it changes none of the paper's comparisons: its safe pair canonicity (13.47%) is below MathNet's (16.53%), only about 5 points above whitespace-only cleaning (8.31%), and far below TeXForm's.


Question 2: Could you compare TeXForm with a repaired existing tool, and report runtime and memory on the same hardware and inputs?

Yes to both, see Weakness 2 above. We repaired all nine documented defects in im2markup and MathNet without adding command knowledge, and neither repaired tool exceeds 18% safe pair canonicity, against 37.15% for TeXForm (Faithful). On the same 23757 UniMER-Test formulas and the same CPU Platform, TeXForm is the fastest LaTeX-parsing normalizer across 16 processes and uses the least memory of all tools in both settings.


We hope the check of every changed decision against rendered images, the repaired baselines, and the cost measurements address your concerns about the strength of our evaluation contribution and the benefit beyond repairing existing tools. If so, we would be grateful if you would take them into account in your final assessment.

AuthorsOpenReview ↗

Follow-up to Reviewer WZKC

Dear Reviewer WZKC,

Thank you again for your review. With about one day left in the discussion period, we would like to check whether our response resolves your concerns. Below is a brief summary of our answer to each point, including the experiments you requested.

Evaluation contribution. You expected work in this area to reveal important new evaluation problems or to provide a validated, more trustworthy evaluation protocol. Our paper does both. It shows that the official scoring of three widely used benchmarks rejects 19–26% of predictions that render identically to their references (Table 2). It also provides a protocol that checks each scoring decision against images rendered from the raw strings (Sections 4 and 6.4). Under this check, replacing the official normalizer on UniMER-Test with TeXForm turns 12,278 false rejects into accepts and loses 7 true accepts, so the ranking change follows from verified corrections rather than from the scores alone.

Repaired tools. As you suggested, we repaired all nine documented defects in im2markup and MathNet and audited the repaired versions (new Appendix C.7). They become about one point safer but fold no more pairs than their originals. They fold at most 17.45% of render-equivalent pairs without changing the rendering, against 37.15% for TeXForm (Faithful).

Runtime and memory. As you requested, the new Appendix G measures all tools on the same hardware and inputs. TeXForm normalizes one million formulas in about 12 seconds on 16 processes, with the lowest peak memory of all tools.

Uni-MuMER's cleaner. It does behave differently from MathNet, so we added it to the audit (Table 12). It folds 13.47% of render-equivalent pairs without changing the rendering (below MathNet's 16.53%), so including it does not change the paper's comparisons.

We would appreciate knowing whether these results resolve your concerns, or what remains open, so that we can respond before the discussion closes. If your concerns are resolved, we would be grateful if your final assessment could reflect this.

Best regards,

The Authors

Reviewer_tZH8OpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes (1) a formalization of the problem of normalizing a LaTeX formula to a canonical shape, and (2) the TeXForm algorithm for performing such normalization. The algorithm is evaluated on three settings; raw properties of the normalizer, its effect on dataset construction (existing datasets are not faithful to the source formulas), and the effect on scoring of other papers that require text-to-text matching for LaTeX formulas (the ranking flips based on the normalizer).

Critical strengths

An interesting problem formalization, especially because of the relation between text and rendered images. Experiments show that all four properties of the normalization are non-trivial to guarantee. The problem is also relevant, as the paper exposes potential flaws in existing datasets and results by changing how formulas are normalized.

The approach uses LLMs for offline knowledge construction, but can then be applied using a symbolic parser. In a highly predictable and structured domain, I appreciate such knowledge distillation for symbolic approaches. The knowledge base can be continuously expanded (higher canonicity and coverage) and is interpretable, which helps in retaining the safety and idempotence.

Critical weaknesses

The number of rewrite rules seems small, and there is no ablation on how relevant each rule is. I would be interested to see if these 151 are all core and common rewrites, or if there is a long tail of rewrites that needs to be slowly expanded?

Questions for authors
  • Did you try using the agent panel to suggest additional rewrite rules?
  • Do you have statistics of how often each rule is triggered on the different experiments?
5 comments
AuthorsOpenReview ↗

Response to Reviewer tZH8 (1/2): Rule Count, Per-Rule Statistics, and Ablation

Thank you for your thoughtful review and positive assessment of our formalization and of distilling LaTeX knowledge offline with LLMs for deterministic run-time execution.

In short, rule-based canonicalization is driven by a small core of frequent rules, substantial canonicalization already happens outside the rewrite rules, and the 151 rules already cover a long tail of rare syntax (Section 5; Appendix B.7; new Appendix B.9).


Weakness 1: Rule Count and Per-Rule Relevance

Per-Rule Statistics and Leave-One-Out Ablation

Appendix B.7 (Table 7) already ablates whole pipeline stages. Disabling all rewrite rules lowers Faithful's pair canonicity from 37.50% to 16.47%, so the rules account for 6,210 of its 11,073 folded pairs (Appendix B.9).

Following your suggestion, we have now also profiled how often each rule applies on all five evaluation supports and run a leave-one-out ablation on the 29,528 equivalent pairs (new Appendix B.9, Tables 10–11).

Across all five supports, the top 10 rules account for 94.5%–98.4% of all Faithful applications (Table 10) and the top 30 for over 99.9%, whereas nearly a third of the enabled Faithful rules handle rare notations such as \above or \idotsint and never apply here (Appendix B.9).

The same concentration holds rule by rule, both in how often each rule applies and in the pairs that stop folding when that rule alone is disabled (Table 11):

RuleRaw labels (% of applications)Model predictions (% of applications)Pairs lost when disabled
a \over b → \frac{a}{b}33.10.05
\ → ~26.915.71,085
^{\prime} → '15.347.6573
\rightarrow → \to7.77.81,756
\le → \leq3.33.8619
\ge → \geq2.31.3315
\dots → \ldots / \cdots1.33.0769
\operatorname{sin} → \sin0.113.5894
Others (31 / 24 rules applied)10.07.2-

These eight rules account for 90.0% of Faithful applications on raw labels and 92.8% on model predictions. A rule's frequency on raw labels does not predict its contribution on model outputs: legacy \over is the most frequent rule on raw labels but almost never appears in model outputs, whereas \operatorname{sin} → \sin is rare in raw labels yet folds 894 pairs.

Why 151 Rules Are Less Modest Than They Appear

The 151 rules may look few next to the 1,213 knowledge-base records, but their count understates how much TeXForm canonicalizes, for four reasons:

  • Most knowledge-base records need no rewrite rule. The parser needs all 1,213 records to read diverse formulas accurately, but most symbols and commands have no equivalent form to fold.
  • Much of TeXForm's canonicalization happens outside Rewrite. Parsing against the knowledge base's argument specifications and serializing the tree canonically already fold 12.54% of the pairs, since \frac12 and \frac{1}{2} yield the same tree. Rule-independent transform phases that unify style commands (e.g., {\bf x} → \mathbf{x}) and remove redundant braces raise this to 16.47% with all rewrite rules disabled, about 44% of Faithful's 37.50% (Section 5; Appendices B.7 and B.9).
  • A single AST rule can cover an entire command family. Rules operate on syntax trees and command metadata rather than literal strings, so one parameterized rule folds all 32 standard functions in \operatorname{...}.
  • 151 AST rules already cover far more equivalences than existing normalizers. MathNet defines only 44 non-identity token aliases, and im2markup relies on roughly 10 string replacements. We estimate that, written as flat regexes in their style, TeXForm's family-level rules and per-command structural handling would amount to hundreds to over a thousand individual rules.
AuthorsOpenReview ↗

Response to Reviewer tZH8 (2/2): Direct Answers to Questions

Question 1: Did you try using the agent panel to suggest additional rewrite rules?

Not yet for additional rules, but the annotation panel already proposed most of the current ones (Appendix B.5). When our three-model annotation panel analyzed the MathJax source, it also recorded source-level aliases, desugarings, and discouraged forms that directly seeded 85 of our 151 rules, and agents helped catalog documentation candidates for the remaining 66.

We intentionally did not build an open-ended agentic harness to maximize rule count. The remaining candidates mostly target rare notations, and deciding canonical forms and profile contracts calls for careful human review of every rule. Meanwhile, guided by corpus statistics, we have continued expanding both the knowledge base and the rule set since submission.


Question 2: Do you have statistics of how often each rule is triggered on the different experiments?

Yes, in the new Appendix B.9. Table 10 reports how often each rule applies on all five evaluation supports, and Table 11 adds a leave-one-out ablation on the 29,528 equivalent pairs. Rule applications are highly concentrated: the top 10 rules account for 94.5%–98.4% of all Faithful applications. See Weakness 1 above for the per-rule breakdown.

Thank you again for your encouraging review. The per-rule analysis in the new Appendix B.9 was directly motivated by your questions.

AuthorsOpenReview ↗

Follow-up to Reviewer tZH8

Dear Reviewer tZH8,

Thank you again for your review. With about one day left in the discussion period, we would like to check whether our response answers your questions.

You asked whether the 151 rules are all core rewrites or a long tail, and how often each rule applies. Following your suggestion, the new Appendix B.9 (Tables 10–11) reports per-rule statistics on all five evaluation sets, together with a leave-one-out ablation. The answer is both: the top 10 rules account for 94.5%–98.4% of Faithful's rule applications, and the remaining rules form a long tail that covers rare syntax, while parsing, serialization, and rule-independent phases already fold 16.47% of equivalent pairs before any rule applies.

If anything remains unclear, we would be glad to clarify it in the remaining time.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response and Summary of Revisions

We thank the Area Chair and Reviewers tZH8, WZKC, and 99Jz for their thoughtful and constructive feedback.

We have uploaded a revised manuscript with all changes marked in blue, using the extra discussion-period page (9 to 10 main-text pages) to improve clarity while placing new experimental analyses in the appendices. Alongside our point-by-point replies, we summarize the main review concerns and all manuscript revisions below.


1. Main Concerns and How We Addressed Them

  • Are the 151 rules all core rewrites, or is there a long tail? (tZH8 W1 & Q2) Both. The top 10 rules account for 94.5%–98.4% of Faithful's rule applications, a long tail covers rare syntax, and parsing, serialization, and rule-independent phases already fold 16.47% of equivalent pairs before any rule applies (Section 5; Appendix B.7; new Appendix B.9, Tables 10–11).
  • What is the evaluation contribution beyond score and ranking shifts? (WZKC W1) We reveal a new evaluation problem and provide a rendering-based protocol that audits it and validates fixes (Section 4). Benchmark scoring itself wrongly rejects 19–26% of render-equivalent predictions (Table 2). With each benchmark's scoring rule fixed, we check every changed decision against raw renderings. On UniMER-Test, TeXForm recovers 12,278 false rejects, loses only 7 true accepts (6 on inputs it declines to normalize), and reduces accepted pairs that render differently from 220 to 163 (Section 6.4). The ranking reversal is a consequence of these verified corrections.
  • Does TeXForm's benefit go beyond repairing existing tools? (WZKC W2 & Q2) Yes. Repairing all nine documented defects in im2markup and MathNet raises safety by only about 1 point, folds no additional pairs, and leaves safe pair canonicity (defined below) below 18%. The gap therefore comes from missing command knowledge rather than isolated bugs (new Appendices C.7–C.8, Tables 14–15).
  • What does TeXForm cost at scale? (WZKC W2 & Q2) Across 16 processes, TeXForm normalizes over 80,000 formulas/s (~12 s per million formulas), the fastest of all LaTeX-parsing normalizers in that setting, with the lowest peak memory of all audited tools (36 MiB on one core; new Appendix G, Table 37).
  • What is the methodological novelty beyond the parse–rewrite–serialize pipeline? (99Jz W1) The pipeline itself is standard. Our contribution is to turn ad-hoc preprocessing into a formally specified, auditable ML component by (1) defining four checkable rendering-based properties and when normalized matching is valid (Proposition 1, Section 4), (2) exposing silent corruption across datasets and benchmarks independent of TeXForm (Sections 3, 6), and (3) driving AST rewriting with a human-adjudicated command knowledge base derived from MathJax and explicit preservation contracts rather than hand-written patterns (Section 5).
  • Do the folded pairs keep their original rendering? (99Jz W2 & Q1; WZKC Q1) Almost always for TeXForm. Only 0.91% of the pairs TeXForm (Faithful) folds change the rendering, against 75.09% for MathNet. A new Table 1 column, safe pair canonicity, counts a fold only when the output renders like both inputs, testing both conditions of Proposition 1 on the same pairs. On this measure, TeXForm reaches 37.15% (Faithful) and 45.96% (Corpus), whereas no existing tool exceeds 20%, including the newly audited Uni-MuMER cleaner (new Appendices C.2 and C.8, Tables 12 and 15).

2. Readability and Terminology Revisions

  • Clarified the method and formalization (Abstract; Sections 1–5, 7) by adding a Normalization profiles paragraph, expanding the parsing and rule-independent transform descriptions with a walkthrough of Figure 2b, and adding a counterexample in Section 4.
  • Restructured Section 6 and Tables 1–2 (99Jz Presentation) by adding an upfront Supports and rates paragraph (Section 6.1), replacing Table 1's Role column and dagger marks with prose (Section 6.2), switching Table 2 to a side-by-side Official vs. TeXForm layout, and reporting recovered false rejects, lost true accepts, and false-accept candidates per track (Section 6.4).
  • Unified terminology (normalizer, fold, safety, render-equivalent, rendering-equivalence criterion, paired-label fidelity), defined specialized terms on first use (render drift; trigger vs. apply), and stated decision rules in Appendix A.3. Five new appendix tables (10–11, 14–15, 37) shift former Tables 10–11 to 12–13 and 12–32 to 16–36.

3. Corrections to Reported Numbers

  • Corrected TeXForm's OCRBench column in Table 2 (Section 6.4; Appendices E.3–E.4, F.1). Stripping outer math delimiters such as $$ before normalization, as already done on UniMER-Test, resolves the 28% of equivalent pairs previously left without a verdict.
  • Updated three Table 1 entries to the final run (also reflected in Tables 12 and 14), each by at most 0.06 points.

All other reported numbers are unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.