acceptodds
Under review as a conference paper at ICLR 2027

Structure-Aware Equivalence for Evaluating Autoformalized Statements

Abstract

Evaluating autoformalized mathematical statements requires determining whether a candidate preserves the mathematical claim intended by a trusted reference. Existing methods approach this problem from complementary directions: reference-free learned evaluators estimate semantic agreement but are not constrained to a formal equivalence relation, while reference-based proof methods typically employ whole-statement logical equivalence to provide formal certificates, which is too coarse for translation fidelity and may identify mathematically unrelated statements. We introduce Structure-Aware Equivalence (SAE), a role-preserving formal equivalence relation between syntactic and whole-statement logical equivalence that allows formally justified reformulations while preserving translation-relevant mathematical structure. We operationalize SAE in the Autoformalization Correctness Evaluator (ACE), which provides certified equivalence or inequivalence judgments and reports Inconclusive otherwise. On ProofNetVerif, a benchmark introduced with BEq+, ACE retains substantial equivalent-case coverage while remaining highly conservative in certifying disputed or inequivalent pairs. On targeted diagnostics, BEq+ accepts 94–96% of unrelated but whole-statement logically equivalent pairs, while Mathesis LeanScorer accepts 42% of localized meaning-changing perturbations; ACE accepts none of either set as equivalent.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
AI for Math, Autoformalization, Formal Verification, Semantic Equivalence, Evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2Dr9OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies the problem of determining whether two Lean formalizations are semantically equivalent.

Given two Lean formalizations, the goal is to determine whether they express the same mathematical statement.

A previous approach, BEq, checks equivalence by using a theorem prover to establish both directions of implication. Specifically, it assumes one statement and attempts to prove the other, then reverses the direction. If both implications are successfully proved in Lean, the statements are declared equivalent.

However, this approach can produce false positives. In particular, two mathematically unrelated statements may be declared equivalent if both are independently provable. A sufficiently powerful prover can establish each statement without relying on the other, thereby proving their logical equivalence even though they express different mathematical claims.

To alleviate this issue, the authors propose Structure-Aware Equivalence (SAE), implemented through the ACE framework.

The approach works as follows.

First, each Lean statement is normalized into a logical form consisting of a quantifier prefix, a sequence of hypotheses, and a conclusion. This involves a revert-like transformation of theorem arguments, extracting admissible quantifiers into an outer prefix, identifying the top-level hypotheses, and treating the remaining expression as the conclusion.

Next, ACE enumerates admissible alignments between the variables of the two statements. Each alignment must preserve variable types, quantifier kinds, and dependencies. For statements containing hypotheses, ACE then checks whether the conjunctions of their hypotheses are logically equivalent and whether their conclusions are logically equivalent under the same variable alignment. Both equivalences must be established independently of the original hypotheses. For statements without hypotheses, ACE instead compares the relational structure and corresponding operands of the conclusions.

If at least one admissible alignment satisfies all the required conditions, the statements are declared SAE-equivalent. If all alignments are ruled out, they are declared SAE-inequivalent. Otherwise, the result is inconclusive.

Experimental results show that ACE reduces false positives compared with existing approaches, particularly for unrelated but independently provable statements. However, this improvement comes at the cost of rejecting some genuinely equivalent formalizations. Moreover, since SAE still relies on logical equivalence within individual components, it mitigates rather than fundamentally eliminates the shared-provability problem.

Critical strengths
  • The proposed approach can detect certain edge cases, such as mathematically unrelated but logically equivalent statements, that previous approaches fail to distinguish.
Critical weaknesses
  • One criticism of BEq is that it may incorrectly classify mathematically unrelated statements as equivalent. However, I am not convinced that the proposed approach fundamentally change the situation. For example, consider:
theorem thm1 (a b : ℕ) (h : a + 0 = b) : a = b := by
  sorry

theorem thm2 (a b : ℕ) (h : a * 1 = b) : a = b := by
  sorry

After normalization, ACE can still prove the equivalence of the two hypotheses, since both a + 0 = b and a * 1 = b are equivalent to a = b. The conclusions are also identical. Thus, ACE would classify these statements as equivalent despite their different mathematical expressions. This suggests that the proposed method mitigates, rather than fundamentally eliminates, the limitations of BEq.

Questions for authors

see weaknesses

3 comments

We thank the reviewer for the constructive feedback and welcome further questions.

The fundamental problem is specifying translation correctness before attempting to verify it. Syntactic equivalence is a precise but overly restrictive criterion: it rejects algebraic rewrites that preserve mathematical meaning. SAE instead explicitly specifies which mathematical roles must be preserved. Our goal is not to reproduce every human judgment, but to define an explicit equivalence criterion whose guarantees and empirical behavior can be systematically analyzed.

The proposed counterexample

The proposed example is correctly classified as equivalent under SAE's intended preservation policy. For all a,b∈Na,b\in\mathbb N, a+0=b  ⟺  a=b  ⟺  a⋅1=b.a+0=b \iff a=b \iff a\cdot1=b. Both hypotheses express exactly the same predicate over the same mathematical objects, and the conclusions are identical. SAE deliberately accepts such algebraic reformulations. Rejecting them would require a stricter, expression-sensitive criterion that also rejects legitimate rewrites, such as Odd n versus ¬ Even n in the ProofNetVerif equivalent case exercise_1_27.

The example illustrates why correctness must be specified before it can be certified. Whether translation fidelity requires preserving the original arithmetic expressions or merely the mathematical conditions they express is a choice of equivalence criterion. Human experts may reasonably differ on this selection. Neither follows automatically from the formal statements. SAE deliberately adopts the latter, more permissive policy within matched mathematical roles. By making this choice explicit, SAE allows its guarantees and limitations to be analyzed independently of proof-search strength. We do not claim that SAE captures every human interpretation or eliminates every shared-provability ambiguity.

Thank you for the response. I agree that my original example may not have been the most appropriate illustration of my concern.

However, unless I have fundamentally misunderstood the proposed approach, I believe my broader concern remains. I understand that SAE deliberately adopts a particular equivalence criterion rather than attempting to capture every human interpretation, and that this helps address certain failure modes of existing approaches. Nevertheless, I remain unconvinced that relying on component-level logical equivalence fundamentally resolves the shared-provability issue that motivates the paper.

Thank you for the follow-up. We would like to assert that component-level logical equivalence in SAE is substantially more restrictive than equivalence of complete theorem statements.

Under a single admissible alignment, SAE requires the corresponding premises and conclusions to be equivalent for every assignment of the aligned variables. Crucially, conclusion equivalence must hold independently of the premises, different from the whole-statement equivalence.

For example, consider:

∀n:Z,n>0→n≥1\forall n : \mathbb{Z},\\ n > 0 \to n \ge 1

∀n:Z,n>0→0=0\forall n : \mathbb{Z},\\ n > 0 \to 0 = 0

Both statements are provable, and their binders, types, and premises are identical. However, SAE rejects their equivalence: the conclusions n≥1n \ge 1 and 0=00 = 0 are not equivalent for every integer nn (e.g., n=0n = 0). This distinction cannot be obtained through basic binder or type alignment alone.

Our new controlled experiments for reviewer MGZj confirm this difference. We constructed 30 pairs with matching variables, types, dependencies, and premises. On 10 premise–conclusion leakage cases, BEq accepted all 10 as equivalent, whereas ACE certified 4 as inequivalent and returned Inconclusive for the other 6, with no false equivalence judgments.

We also explicitly tested the remaining limitation. Among 10 pairs with distinct independently provable conclusions, ACE accepted 9. A stricter recursive comparator prevents some such acceptances, but also rejects legitimate reformulations: on 10 faithful-rewrite cases, ACE accepted all 10, whereas the stricter comparator accepted only 1. On 257 naturally occurring premise-bearing pairs already certified equivalent by ACE, the stricter comparator retained 228 as equivalent, rejected 3, and left 26 inconclusive.

SAE therefore does not universally eliminate shared provability, but its role-aligned, assignment-uniform equivalence provides a substantive restriction beyond whole-statement logical equivalence. The experiments make both its benefits and its remaining limitations explicit. Our contribution is to specify and certify this intermediate relation, rather than claim a universal solution to mathematical-intent ambiguity.

Reviewer_MGZjOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Structure-Aware Equivalence (SAE), a new equivalence criterion for evaluating the correctness of autoformalized mathematical statements. The authors argue that whole-statement logical equivalence is insufficient for assessing translation fidelity, since mathematically unrelated statements may still be logically equivalent. To address this issue, SAE introduces structural constraints that preserve variable correspondence, premise-conclusion roles, and relational structures.

Critical strengths
  1. The paper identifies an important limitation of using whole-statement logical equivalence to evaluate autoformalization correctness.

  2. The proposed SAE framework provides an explicit and formally defined intermediate equivalence relation between syntactic equivalence and whole-statement logical equivalence. The attempt to preserve mathematical roles is conceptually interesting.

Critical weaknesses

While the problem is important and the proposed framework is interesting, I have significant concerns regarding the theoretical justification of SAE and the adequacy of the experimental evaluation. In particular, the proposed equivalence criterion does not fundamentally resolve the shared-provability problem that motivates this work.

W1. The proposed SAE criterion does not fundamentally resolve the shared-provability problem.

My primary concern is that the core design of SAE does not fully address the fundamental issue motivating this paper. Their proposed Premise-Conclusion Equivalence (Definition 2.1) still relies on unrestricted proposition-level logical equivalence within the premise and conclusion components. Specifically, given an admissible variable alignment, the method requires: 1,Logical equivalence between the conjunctions of premises; 2,Logical equivalence between the conclusions. Consequently, the same shared-provability problem identified for whole-statement equivalence can reappear within individual components. This exposes a fundamental limitation: SAE restricts where logical equivalence is applied, but does not ensure that logical equivalence within the resulting components preserves mathematical meaning. The authors should clarify the theoretical justification for treating unrestricted logical equivalence of premises and conclusions as sufficient evidence of translation fidelity.

W2. The diagnostic experiments are biased toward the proposed method and fail to establish that SAE resolves the shared-provability problem

The design of the Unrelated Shared-Provability Set appears to favor the proposed method. The authors construct 50 examples in which two unrelated statements are logically equivalent because their complete propositions are independently provable. However, the examples provided in Appendix E include cases such as comparing a statement about natural numbers with one about sets, or comparing Not False with a natural-number reflexive equality. Such examples can be rejected by ACE through its type-preservation and variable-alignment constraints, without demonstrating that SAE can distinguish mathematically different claims when their structures are otherwise compatible. Consequently, the reported 0% false-positive rate on this diagnostic set may primarily reflect the ease with which these examples violate the method's explicit structural constraints, rather than the effectiveness of SAE in addressing the underlying shared-provability problem. This raises concerns about whether the diagnostic set provides a sufficiently balanced and rigorous evaluation of the proposed method. More importantly, the current experiments do not establish that SAE can handle the more challenging case that motivates the paper: two statements have the same quantified variables, compatible types, identical premises, and independently provable conclusions, yet express different mathematical claims. In fact, the formal definition of SAE appears unable to distinguish at least some such cases.

W3. The high rate of certified inequivalence on genuinely equivalent statements raises concerns about practical utility.

According to Table 1, ACE certifies only 55.4%–60.0% of the Consensus-Equivalent subset as equivalent, while classifying approximately 29.4%–30.5% as certified SAE-inequivalent. Such a result indicates that the proposed equivalence criterion rejects a substantial proportion of statement pairs considered faithful translations under the paper's own human audit. The authors attribute many of these cases to the conservative alignment policy and structural restrictions of SAE. However, this creates an important trade-off: a useful autoformalization evaluator should avoid false equivalence judgments while accepting legitimate mathematical reformulations. The current results suggest that SAE may achieve conservativeness partly by imposing restrictions that are too strong for practical translation evaluation. The authors should provide a more systematic analysis of these false negatives and clarify whether this level of disagreement with human judgments is acceptable for the intended applications, particularly benchmark evaluation and training-data selection.

W4. The normalization procedure requires a clearer correctness justification.

The proposed framework depends heavily on the restricted prenex-style normalization described in Section 2 and implemented in Section 3.1. However, the paper does not sufficiently specify the exact normalization transformations and their correctness guarantees.

Questions for authors

Q1:

There exists some substantial difference between the two equivalence criteria introduced in Section 2. For premise-free statements, SAE requires structural matching of relational expressions, including compatible relation operators and equality of corresponding operands. This prevents two unrelated but provable relational statements from being classified as equivalent. However, for premise-bearing statements, the conclusion is compared using unrestricted proposition-level logical equivalence, without imposing the same relational structure constraints. This raises a fundamental question about the coherence of the proposed criterion: why should the presence of an implication substantially change the required level of structural preservation for the conclusion? Why not investigate applying structure-aware comparison recursively to the conclusions of premise-bearing statements?

7 comments

We thank the reviewer for the constructive feedback and welcome further questions.

SAE addresses a foundational specification problem, not a universal solution to mathematical-intent ambiguity. A prover can correctly certify whole-statement logical equivalence while accepting unrelated mathematical claims, which stronger proof search cannot repair. SAE instead specifies which mathematical roles must correspond, while allowing broad reformulations within matched roles. Its contribution is an explicit correctness criterion, a verification procedure, and a way to study their guarantees and empirical tradeoffs.

W1/Q1. Shared provability and the granularity of SAE

The reviewer's observations identify the granularity of SAE's specification, rather than an inconsistency in its verification guarantees. SAE deliberately preserves correspondence between mathematical roles while treating matched premises and conclusions as semantic units. Their equivalence must hold uniformly over aligned variables, independently of the premises. This permits substantial algebraic, logical, and definitional reformulations.

For example, SAE accepts the following rewrite in the conclusion ∀n:Z,n>0→n≥1⟺∀n:Z,n>0→(n>1∨n=1),\forall n:\mathbb Z, n > 0 \rightarrow n\ge 1 \quad\Longleftrightarrow\quad \forall n:\mathbb Z, n > 0 \rightarrow(n>1\lor n=1), but rejects the equivalence to ∀n:Z,n>0→0=0\forall n:\mathbb Z, n > 0 \rightarrow 0=0, since ∀n:Z,n≥1↔0=0\forall n:\mathbb Z,\quad n\ge1\leftrightarrow 0=0 is false. However, this will be accepted by whole-statement logical equivalence since both are true. Thus, SAE is more restrictive than whole-statement logical equivalence, while still allowing ordinary mathematical rewrites within an aligned role for all possible variable assignments.

We agree that shared provability may remain within matched components. This establishes a boundary of SAE's chosen granularity, not a contradiction in its definition. Recursively imposing structural correspondence would define a stricter relation, but would also reject additional legitimate reformulations.

For premise-free relational statements, SAE preserves expressions and relation operators; for premise-bearing statements, it preserves the coarser premise–conclusion organization. Our contribution is to make this preservation policy explicit, so that its formal guarantees and empirical suitability can be evaluated separately. We will clarify that SAE mitigates shared-provability failures at this granularity rather than eliminating them universally.

W2. Diagnostic-set design

Our diagnostic sets are designed to isolate specific failure mechanisms, while ProofNetVerif evaluates broader practical behavior. The Unrelated Shared-Provability Set contains mathematically unrelated statements with provable whole-statement equivalence. The Semantic Perturbation Set contains localized meaning changes within largely unchanged structures. Their inequivalence labels are determined independently of ACE's predictions.

These sets represent relatively uncontroversial inequivalences and common failure patterns in LLM autoformalization. These controlled tests examine whether evaluators reject clear semantic mismatches that whole-statement equivalence or similarity-based scoring may overlook. They are not intended to estimate general accuracy or demonstrate that SAE rejects every possible shared-provability case.

We separately evaluate on ProofNetVerif, a general-purpose benchmark introduced with BEq+ rather than constructed for SAE. The experiments serve complementary purposes: the diagnostics isolate specific criterion-level failures, while ProofNetVerif measures broader empirical behavior. We will clarify this distinction.

W3. Equivalent-case coverage

Practical utility depends on the guarantees required by the application, not human-label agreement alone. Human-label agreement measures predictive fidelity, whereas formal certification requires a precise specification of what an accepted equivalence guarantees. These are distinct objectives.

Explicit correctness specifications are essential for auditable evaluation. For applications such as trusted dataset curation and reproducible model benchmarking, an interpretable acceptance criterion is valuable beyond predicting human agreement. SAE provides such a criterion while accepting substantial classes of legitimate reformulations.

Our experiments quantify the tradeoff. ACE certifies approximately 57–59% of Consensus-Equivalent pairs as Eq, while accepting only 0.3% of Disagreement-Equivalent pairs and 0.1–0.2% of the original Inequivalent set. Its conservative policy also rejects some faithful reformulations. Representative disagreements of SAE on the Consensus-Equivalent subset arise from the misalignment of roles, including cases like:

a. Function equality versus pointwise equality: ∃ c, f = fun x => c versus ∃ c, ∀ x, f x = c.

b. Represent auxiliary variables by an equality: Irreducible (X^6 + 30*X^5 - 15*X^3 + 6*X - 120 : Polynomial ℤ) versus (p : Polynomial ℤ) (h : p = X^6 + 30*X^5 - 15*X^3 + 6*X - 120) : Irreducible p.

c. Existence versus set nonemptiness: ∃ x, ∀ n, x ∈ G n versus (⋂ n, G n).Nonempty.

SAE does not aim to replace human judgment in every application. It provides a conservative, auditable certification policy, whose suitability depends on the relative costs of false acceptance and missed equivalence. The approximately 30% audit disagreement characterizes this tradeoff; it does not, by itself, establish a lack of practical utility.

W4. Normalization correctness

SAE is designed to be agnostic to the languages (Lean, Rocq, etc), while the normalizers are inevitably language-specific. Our normalizer in Lean handles Lean 4’s various features, like typeclass-instance binders ([x : typeA]), implicit binders ({x : typeA}), and subtypes ({x : α // P x}).

For the supported fragment, normalization:

  1. collects outer universal binders, including implicit and typeclass binders, while preserving dependencies;

  2. decomposes implications into ordered premises and conclusions;

  3. applies α-renaming to bound variables and universe names; and

  4. canonicalizes elaborated constants and instance expressions without commuting or reordering dependent binders.

The normalization rules are equivalence-preserving under their stated preconditions. We will document these rules and their correctness arguments for the supported fragment. The implementation of normalization is not independently machine-verified and remains part of ACE's trusted implementation boundary.

Thank you for the response and clarifications. Some of my concerns have been partially alleviated, but my major concerns remain.

W1/Q1: Justification of the chosen granularity. The authors acknowledge that shared provability remains within matched components and characterize this as a deliberate choice of granularity. However, the paper still lacks theoretical justification or comparative evidence supporting this choice for translation fidelity, including the different treatment of conclusions in premise-bearing and premise-free statements. The argument that recursive structural comparison would reject more legitimate rewrites is reasonable, but which important rewrites would be lost, which false acceptances would be prevented, and what would the resulting trade-off be? Observing that a stricter relation accepts fewer pairs does not, by itself, justify the current boundary.

W2: Diagnostic evaluation. Clarifying the complementary purposes of the diagnostic sets and ProofNetVerif is helpful, but does not resolve the concern about diagnostic coverage. The response provides no results for the challenging cases identified in my review: matching variables and types, identical premises, and distinct independently provable conclusions. A breakdown of rejection reasons and an ablation isolating the contribution beyond basic structural filtering are also missing.

W3: Practical utility. I agree that a conservative, auditable criterion can be useful. However, this does not establish that the current trade-off is suitable for benchmarking or training-data selection, particularly when approximately 30% of Consensus-Equivalent pairs are certified as SAE-inequivalent. The examples help explain these disagreements, but a systematic quantitative analysis and justification for the intended applications are still needed.

W4: Normalization correctness. The clarification of the trusted implementation boundary partially alleviates this concern. However, the response outlines broad operations and promises further documentation; the exact normalization rules, their preconditions, and correctness arguments for the supported fragment still need to be provided.

Reviewer_fLmXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+2)
Read review
Paper summary

This paper introduces a novel criterion, structure-aware equivalence (SAE), for determining the semantic equivalence between two formal statements and implements its evaluator, ACE, in Lean 4. It requires that binders, variables, premises, and conclusions have corresponding counterparts across the two statements. Experiments demonstrate the effectiveness of the proposed criterion compared with baselines.

Critical strengths
  1. SAE demonstrates empirical improvement over baselines on ProofNetVerif.
Critical weaknesses
  1. Representation sensitivity. SAE relies on a representation-sensitive separation of binders, premises, and conclusions. However, equivalent formalizations can move constraints between hypotheses and dependent types, e.g., ∀ x : ℝ, x > 0 → P x and ∀ x : {x : ℝ // x > 0}, P x.val.
  2. Lack of comparison. ASSESS[1] also combines semantic and structural information for evaluating similarity between two formal statements, but is neither discussed nor evaluated.

[1] ASSESS: A Semantic and Structural Evaluation Framework for Statement Similarity. ICLR 2026. https://openreview.net/forum?id=avwNGWtiHF

Questions for authors
  1. Can the restricted prenex-style normalization handle such flexibility between hypotheses and dependent types?
  2. Could the authors estimate how often such transfers of constraints between hypotheses and dependent types lead to false negatives?
  3. Could the authors include [1] for discussion and comparison?

[1] ASSESS: A Semantic and Structural Evaluation Framework for Statement Similarity. ICLR 2026. https://openreview.net/forum?id=avwNGWtiHF

2 comments

We thank the reviewer for the constructive feedback and welcome further questions.

SAE specifies which mathematical equivalences should be certified, while structural similarity alone does not. This is the central distinction from ASSESS. SAE makes preservation of mathematical roles a condition of equivalence, rather than relying on similarity or proof-search behavior to preserve them.

1. Representation sensitivity

SAE is designed to be language-agnostic (Lean, Rocq, etc), while normalizers are inevitably language-specific. Our Lean-specific normalizer handles Lean 4 features including typeclass-instance binders ([x : typeA]), implicit binders ({x : typeA}), and subtypes of the form {x : α // P x}.

The specific subtype example is handled by our supported normalization. The statement ∀x:y:R∣y>0,P(x.val)\forall x : \\{y : \mathbb R \mid y>0\\}, P(x.\mathrm{val}) is effectively normalized to ∀y:R,y>0→P(y)\forall y : \mathbb R, y>0\to P(y). The normalizer extracts the subtype constraint as a premise and aligns the underlying carrier variable. Arbitrary dependent-type encodings remain outside our scope.

Among 787 Consensus-Equivalent pairs, we identified 48 involving transfers between type-level requirements and explicit hypotheses: 3 subtype-to-predicate transfers and 45 typeclass-instance-to-hypothesis transfers. Two received Noneq outcomes attributable specifically to unsupported transfers (4.2% of these cases, or 0.25% of the full subset). These two cases are attached to the end of this response. The other rejected cases involved multiple simultaneous differences, so their failures cannot be attributed to this mechanism alone.

2. Comparison with ASSESS

Incorporating structural similarity does not, by itself, resolve the correctness-specification problem. We thank the reviewer for bringing ASSESS to our attention. ASSESS combines tree-edit distance with proof-guided transformations, but assigns maximal similarity when proof search establishes whole-statement equivalence, even when the original mathematical roles are unrelated. Thus, structural information guides its similarity metric but does not impose a mandatory constraint on maximal similarity.

SAE addresses this distinction at the specification level. An equivalence certificate must satisfy the prescribed correspondence between mathematical roles; stronger proof search cannot bypass these requirements.

We evaluated ASSESS in Lean 4.7.0 using its published EPLA-ProofNet threshold of 0.38, without recalibration on our datasets. The results below demonstrate the practical consequences of these different evaluation objectives.

DatasetASSESS EqACE EqBEq EqMathesis Eq
Unrelated Shared-Provability6%0%94-96%4%
Semantic Perturbation92%0%0-2%42%
ProofNetVerif Consensus-Equivalent87.9%57.1–59.1%59-68.9%81.4%
ProofNetVerif Disagreement-Equivalent60.3%0.3%1.8-4.4%43.8%
ProofNetVerif Inequivalent43.7%0.1–0.2%0.2-0.7%5.1%

ASSESS accepts more human-audited equivalent pairs on ProofNetVerif, but it also accepts substantially more meaning-changing pairs. For example, it misclassifies cases such as Not False versus 37 = 37 in the Unrelated Shared-Provability Set. On the Semantic Perturbation Set, it classifies 92% of meaning-changing pairs as equivalent at its published threshold, compared with 0% for ACE.

These results illustrate the difference between similarity-based prediction and SAE’s explicitly specified equivalence relation. Since the threshold was selected on EPLA-ProofNet, these results are not threshold-optimized for ProofNetVerif or our diagnostic sets. We will cite ASSESS and include the full evaluation protocol.

The transfer-induced false negative cases

The two clear transfer-induced false negatives are listed below:

  1. Munkres|exercise_22_2b
theorem exercise_22_2b {X : Type*} [TopologicalSpace X] {A : Set X} (r : X → A) (hr : Continuous r)  (h : ∀ x : A, r x = x) : QuotientMap r := by sorry

theorem dummy {X : Type u} [TopologicalSpace X] (A : Set X) (r : X → A) (hr : Continuous r) (hrA : ∀ a ∈ A, r a = a) : QuotientMap r := by sorry

This case transfers the constraint from subtype quantification, x : A, to bounded carrier quantification, a : X with a ∈ A.

  1. Dummit-Foote|exercise_4_4_2
theorem exercise_4_4_2 {G : Type*} [Fintype G] [CommGroup G] {p q : Nat.Primes} (hpq : p ≠ q) (hG : card G = p * q) : IsCyclic G := by sorry

theorem dummy {G : Type*} [CommGroup G] [Fintype G] (h : Fintype.card G = p * q) (hp : Nat.Prime p) (hq : Nat.Prime q) (hqp : p ≠ q) : IsCyclic G := by sorry

This case transfers p q : Nat.Primes into natural-number variables with explicit Nat.Prime hypotheses.

Thank you to the authors for the prompt and comprehensive response. It satisfactorily addresses my concerns, and I will raise my rating to 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.