acceptodds
Under review as a conference paper at ICLR 2027

Spanning, Not Counting: Novelty in Language Generation in the Limit, from Finiteness to Compactness

Abstract

A generator can produce endlessly many distinct sentences and keep saying the same thing, and language generation in the limit, in the model of Kleinberg and Mullainathan, cannot tell the difference: it asks only that outputs be unseen. We ask that each output lie outside the span of everything said so far, for a span given by topics, a matroid, a consequence relation, or any monotone map that contains its argument. The right notion of a small set is then a finite core, a set spanned by finitely many of its own points: for the identity a finite set, for a finitary closure operator a set whose closure is compact in the lattice of closed sets. With finite cores in place of finite sets, the characterizations of Kleinberg and Mullainathan (NeurIPS 2024) and of Raman, Li and Tewari (COLT 2025) hold for every span, at every level when only the examples count as said, and uniformly when the generator's own outputs count too. The uniform price is free or fatal: as many examples as before, or no number at all, as for the paraphrases of one statement. At the other levels the generator's bets are the only obstruction, through a second mechanism that counting cannot see: poisoning, a bet valid for one language that together with examples of another spans it. Poisoning is impossible when the span is locally Noetherian, can be fatal beyond, as in a logic with explosion, and cannot be detected from inside single languages. Matroids, and several matroids at once, are the main instances.

Primary area:
learning theory
Keywords:
language generation in the limit, novelty, learning theory, matroids, closure operators, compactness

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_f3ZsOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

Language generation in the limit is a syntactic condition, in which an algorith must generate strings that are not the same as earlier strings it has seen. But as the submission notes, we are often concerned with the meaning of the string that is being output, and therefore we could ask more strongly that the algorithm produce a string whose meaning has not been seen before. The paper develops this idea through the notion of the "span" of a set of strings, and proves interesting positive results.

Critical strengths

The notion of a span as an operation on sets is a natural way to formalize the issue that the paper is considering.

The theorems in the paper show that the requirement is not too strict to yield positive results for generation. The connection to special cases such as matroid structures is very natural.

Critical weaknesses

Because the paper stays at a fairly abstract level, it is harder to the results via concrete instances. This would not necessarily require experiments in real contexts, but just specific pedagogically chosen instances of the problem where the underlying issues could be made more concrete.

The paper Jiaxun Li, Vinod Raman, and Ambuj Tewari. On generation in metric spaces. arXiv preprint arXiv:2602.07710, 2026a. seems more closely related than many of the other cited works, and so it would be helpful for the paper to discuss the comparison to this work more extensively.

Questions for authors

As it stands, the formalism captures an important issue, but the conclusion remain abstract, with limited connections to the kinds of instances of the problem that generally motivates work on language generation. Can you give some more specific instances where this type of analyses could be applied, and show what conclusions you would draw from it?

Can you say more about the connection to Jiaxun Li, Vinod Raman, and Ambuj Tewari. On generation in metric spaces. arXiv preprint arXiv:2602.07710, 2026a?

3 comments

Response to Reviewer f3Zs (1/2)

Thank you for the review, and for pointing to Li, Raman and Tewari, which is indeed the closest work and deserves more than the submission gives it.

W1 / Q1 (concrete instances and the conclusions they support). Four instances, each with what the theorems say about it:

instancespanconclusion
answers counted up to paraphrase, as NoveltyBench counts outputs up to functional equivalencepartition into classesNo more examples than under plain novelty whenever uniform generation remains possible (Theorem 4.1). If the intersection of two targets is one infinite class and each target meets infinitely many other classes, no fixed number of examples suffices, not even one chosen per target, although the class of these two languages is generatable in the limit (Example 4.4).
a new direction among vectorslinear spanIn Qd\mathbb Q^d every language has a finite core, so no output is both valid and novel once a basis of it has appeared (Lemma 3.1(c),(d)): in finite dimension at most dd rounds can succeed, so the linear instance is informative in infinite rank: classes of coordinate flats are generatable from the first example or not uniformly at all, and in the limit whenever countable (Example E.1).
a statement that does not follow from what was saidlogical consequenceThe safe-zone generator of Theorem 4.1 keeps its outputs inside every consistent language, is never poisoned, and succeeds after Coredim+1\mathrm{Coredim}+1 examples whenever any generator succeeds uniformly. A generator that must guess can be poisoned: one invalid output later contradicted by an example makes every later output a consequence of what was said (Example 5.5).
a new topic and a new style at onceunion of the topic and style closuresNovelty in both respects can be impossible although each respect alone has infinite rank (Proposition 7.2): take all strings on one topic and all strings in one style; once one string of each kind has appeared, every further string repeats a topic or a style.

A version of this table is now Table 2 of the revised PDF (page 10, referenced from Section 8).

Response to Reviewer f3Zs (2/2)

W2 / Q2 (Li, Raman and Tewari, "On generation in metric spaces"). Both papers replace "unseen" by a coarser redundancy relation, and both find that the uniform and non-uniform characterizations of Raman et al. survive when "finite" is replaced by a covering notion (in our case uniformly for every span, and non-uniformly for every span when only the examples count and for locally Noetherian spans otherwise). The overlap is closer than our submission says. Below, εg\varepsilon_g and εa\varepsilon_a are their generator and adversary radii, written ε′\varepsilon' and ε\varepsilon in our Appendix H.

  • In the current arXiv version (v2, October 2026) their dimension asks for a finite cover of the safe zone by εg\varepsilon_g-balls centred inside it. For the ball span Ψ(E)=B(E,εg)\Psi(E)=B(E,\varepsilon_g) this is exactly a finite core.
  • Our paraphrase trap is an instance of their model. With distance 11 between paraphrases and 22 otherwise, and εa<εg=1\varepsilon_a\lt\varepsilon_g=1, the ball span is the partition span and the covering number of a set of examples is its size. Their Example B.3, already in v1, is the same phenomenon. Example 4.4 also records that the class is generatable in the limit, and Theorem 6.1 gives the matroid version (two infinite-rank flats meeting in a plane).

The differences:

Li, Raman, Tewarithis paper
redundancywithin εg\varepsilon_g of some example (pairwise)any monotone span with E⊆Ψ(E)E\subseteq\Psi(E); set-wise
whose strings countthe examplesthe examples and the generator's own outputs
threshold measured byεa\varepsilon_a-covering number of the examplesnumber of distinct examples
universe and languagesseparable metric space; languages of infinite covering numbercountable; languages may have a finite core
generation in the limitnot characterized; a union of two uniformly generatable classes of uncountably many hypotheses on R\mathbb R can fail (v2, Theorem 3.5)characterized for countable classes (Prop. 5.1; Thm. 5.2 when the span is locally Noetherian)

Two of these differences carry results. Set-wise redundancy is what linear span and logical consequence need: a vector far from every earlier vector can still lie in their span. Counting the generator's own outputs is the only source of poisoning, so Example 5.5 and Proposition 5.6 have no counterpart in their setting. By analogy with Theorem 5.3, which we prove only for finitary closure operators, covers centred anywhere correspond to the sets that finitely many strings span, and the example in Appendix H shows why the centres must lie inside. In the other direction, their results on scale sensitivity and doubling spaces, and their asymmetric radii, have no counterpart in ours.

Appendix H of the submission was written against their v1, where the cover centres could lie anywhere in the space; its example shows why the centres must lie in the safe zone, which v2 now requires. The submission's remark that their Theorem 3.1 fails as stated applied to v1 only; the revision now states it as a remark about the v1 definition and notes that their current Theorem 3.1 agrees. The revision updates Appendix H to v2, adds Example B.3 and the embedding above there, and adds a paragraph with this comparison to the related work of Section 1 (in blue).

Thank you for this detailed comparison; I think including it in the paper will help the exposition and the connection to related work.

Reviewer_HgExOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper investigates a new notion of novelty within the language generation framework of Kleinberg and Mullainathan. In standard language generation in the limit, an output by the generator is said to be correct if and only if the output is a string from the target language that has not yet been output by the adversary. This allows for the algorithm to potentially repeat similar strings from the target language. Under the original motivation of the generator being similar to an LLM, this situation is unideal since one would like the LLM to output novel sentences that are not merely restatements of what it has already said. Previously in the literature, there have been many attempts at formalizing this notion of novelty. The most prevalent have been various definitions of density, which seeks to measure whether or not the algorithm is generating from a large proportion of the target language. In this paper, the authors introduce a new notion of span that presents a different viewpoint of novelty. Under this definition a span ff, is any monotonic function that maps each subset SS of the universe to another subset f(S)f(S). At any point, the output by the algorithm is correct if the output lies in the target language, and is not contained in the span of the previous adversary or algorithm outputs. The paper shows various results under different scenarios and for different span functions.

Critical strengths

The general idea of measuring novelty in language generation in the limit through a definition like span seems interesting. This takes a different viewpoint from the existing work that mainly investigates novelty through definitions of density, and has the potential to be a fruitful area of study.

Critical weaknesses

The writing in this paper is extremely difficult to read, which makes it impossible to clearly decipher what the main definitions and theorems are, and to evaluate the correctness of the proofs. At a local level, many of the sentences are incoherent. For example, consider the following two sentences from the abstract “The uniform price is free or fatal: as many examples as before, or no number at all, as for the paraphrases of one statement” and “Poisoning is impossible when the span is locally Noetherian, can be fatal beyond, as in a logic with explosion, and cannot be detected from inside single languages”. There are many more examples of such sentences throughout the paper. At a more structural level, the introduction and Section 2 are not structured in a way that allows for a reader to easily extract what the main theorem statements are.

Questions for authors

None

3 comments

Response to Reviewer HgEx (1/3)

Thank you for the review and for quoting the sentences that fail. We agree that the submitted abstract, Section 1 and Section 2 made the definitions and theorems hard to extract. The revised PDF (changes in blue) rewrites the quoted abstract sentences, adds a boxed list of the main results to Section 1, and states the central notions as numbered definitions in Section 2. So that the results can be judged without the PDF, the model and the main results follow in plain form.

W1a (main definitions and theorems).

Model. A countable universe XX of strings, a class H\mathcal H of infinite languages, and the game of Kleinberg and Mullainathan: the adversary enumerates a target language KK, and the generator outputs one string per round. A span is a map Ψ\Psi on sets of strings that is monotone and contains its argument, E⊆Ψ(E)E\subseteq\Psi(E); Ψ(E)\Psi(E) is the set of strings made redundant by having said EE. Examples: all paraphrases of a statement already made (a partition into topics), the linear span of vectors already said, the logical consequences of what has been said. An output is novel if it lies outside Ψ(Es)\Psi(E_s), where EsE_s is everything said before the output: the examples and the generator's own earlier outputs. The identity span gives the original requirement plus never repeating an earlier output, which changes none of the characterizations.

Two definitions carry the paper (Definition 2.5 of the revised PDF):

  • A set AA has a finite core if A⊆Ψ(C)A\subseteq\Psi(C) for some finite C⊆AC\subseteq A: finitely many of its own strings span it. For the identity this means AA is finite; for a matroid, that AA has finite rank.
  • Ψ\Psi is locally Noetherian if every set that finitely many strings span, wherever those strings lie, also has a finite core.

Main results. The safe zone of a finite sample is the set of strings in every language consistent with it. The core dimension Coredim(H)\mathrm{Coredim}(\mathcal H) is the largest size of a consistent sample whose safe zone has a finite core; for the identity it is the closure dimension Cdim(H)\mathrm{Cdim}(\mathcal H) of Raman, Li and Tewari. The domination dimension Domdim(H)\mathrm{Domdim}(\mathcal H) is defined in the same way with "spanned by finitely many strings" in place of "has a finite core" (Definition 2.6).

  1. Theorem 4.1 (every span). H\mathcal H is uniformly generatable iff Coredim(H)\mathrm{Coredim}(\mathcal H) is finite, and then exactly Coredim(H)+1\mathrm{Coredim}(\mathcal H)+1 distinct examples are needed. Moreover Coredim(H)\mathrm{Coredim}(\mathcal H) is either Cdim(H)\mathrm{Cdim}(\mathcal H) or infinite.
  2. Proposition 5.1 and Theorem 5.2. If only the examples count as said, then for every span: H\mathcal H is non-uniformly generatable iff H=⋃nHn\mathcal H=\bigcup_n\mathcal H_n for an increasing sequence H1⊆H2⊆⋯\mathcal H_1\subseteq\mathcal H_2\subseteq\cdots of classes of finite core dimension, and a countable H\mathcal H is generatable in the limit iff no language has a finite core. If the generator's outputs count too, the same holds whenever Ψ\Psi is locally Noetherian.
  3. Poisoning: Example 5.5, Proposition 5.6, Theorem 5.3. An output outside the target can, together with examples of the target, span the target; we call this poisoning (Section 5). Under local Noetherianity it does no harm (Theorem 5.2). Without it, poisoning can be fatal: with literals under propositional consequence, an output pp followed by the example ¬p\neg p is a contradiction, everything follows, and no later output is novel (Example 5.5). Proposition 5.6 relabels the class of Example 5.5 into a second span in which, as in the first, every set inside a language spans only itself; with own outputs counted, the relabelled class is generatable non-uniformly and in the limit, and the original class is generatable at neither level. Theorem 5.3: for a finitary closure operator the following are equivalent: (i) the span is locally Noetherian; (ii) for every class, uniform generation is possible iff Domdim\mathrm{Domdim} is finite, with threshold Domdim+1\mathrm{Domdim}+1; (iii) for every countable class, generation in the limit is possible iff no language is spanned by finitely many strings.
  4. Theorems 6.1-6.2, Propositions 7.1-7.2. For a matroid the characterizations hold with rank in place of cardinality. A class of two languages separates non-uniform generation from the limit (Theorem 6.1), which no countable class does under plain novelty. The two theories coincide iff the matroid has no infinite set of finite rank (Theorem 6.2). Novelty in several matroids at once (a new topic and a new style) is governed by the sets independent in all of them, and infinite rank in each matroid separately is not enough.

Response to Reviewer HgEx (2/3)

What is new relative to the generation-in-the-limit literature. Result 3 has no counterpart under plain novelty. Once the generator's own outputs count, an invalid output can end novelty for good (Example 5.5). The natural alternative to finite cores, "spanned by finitely many strings from anywhere", gives the right uniform and limit characterizations for a finitary closure operator exactly when it is locally Noetherian (Theorem 5.3); so for uniform generation, and at every level when only the examples count, finite cores are the right notion and domination is not. For matroids, Theorem 6.2 gives the exact condition under which rank novelty and plain novelty coincide. Result 1 extends the uniform characterization of Raman-Li-Tewari to every span, and the number of examples needed then either stays the same or becomes infinite; Result 2 extends the other levels to every span when only the examples count, and to locally Noetherian spans otherwise. Example 4.4, which witnesses the second gap of Theorem 6.1, is an instance of the metric model of Li, Raman and Tewari, and their Example B.3 already shows its non-uniform failure; the revision credits it, and the general response compares with their work.

Checking the proofs. Two facts carry every characterization. If a set has no finite core, no finite subset of it spans it, so a generator whose history stays inside the set always has a novel output (Lemma 3.1(b)). If the safe zone of the examples has a finite core, the adversary can show that core without changing which languages are consistent, and then every output is redundant or lies outside some consistent language (Lemma 3.1(c)). When the generator's own outputs count, its history can leave the set, so escape needs the set not to be spanned by finitely many strings from anywhere; local Noetherianity makes that the same as having no finite core (Theorem 5.2). Start with the proof of Theorem 4.1 (Appendix C); it is under one page, in short named steps. The table at the start of Appendix A says where each main-text result is proved.

W1b (the two abstract sentences, and others like them). Both sentences compress three claims each. The revised abstract replaces them by:

When uniform generation remains possible under a span, it needs exactly as many examples as under the original requirement; it becomes impossible when the safe zone of some examples, the set of strings in every language consistent with them, is infinite but spanned by finitely many of its own strings, as the paraphrases of one statement are. At the other levels the characterizations can fail only because the generator's own outputs count as said: an output outside the target language can, together with examples of the target, span it, which we call poisoning. When the span is locally Noetherian, poisoning does no harm. Beyond such spans poisoning can be fatal: under logical consequence, an invalid output followed by an example that contradicts it implies every string, and no later output is novel. Whether poisoning is fatal or avoidable cannot be decided from the sets inside individual languages alone.

Rewriting it also corrected a statement of the submitted abstract: poisoning is not impossible under local Noetherianity; it can occur, but only for targets with a finite core, which no generator handles anyway, so the revision says that it does no harm (Theorem 5.2). Beyond the quoted sentences, the revision rewrites the related-work paragraph and the paragraph on the generator's own outputs in Section 1; the definition of poisoning and the passages on the domination form and the master keys in Section 5; and single sentences in Sections 1-4 and 6-8, including every use of "free or fatal" and the slogan headings of Section 1. It updates the matching terms in the appendix and figures. The appendix proofs are otherwise unchanged; Appendix F.4 and Table 2 are new, and Appendix H is updated to the current version of Li, Raman and Tewari.

W1c (structure of Section 1 and Section 2). After the model, Section 1 now has a paragraph defining finite cores, the core dimension and local Noetherianity, followed by a boxed list (M1)-(M4) of the four results above with their theorem numbers. Section 2 now opens with a one-sentence guide to the definitions the theorems use and states the span (Definition 2.1) and finite cores and local Noetherianity (Definition 2.5) as numbered definitions; in the submission these two were running text, while the games and the core dimension were already numbered. The safe zone is defined in Section 2.1.

Response to Reviewer HgEx (3/3)

S1 (definition of a span, in the summary). A span also contains its argument, E⊆Ψ(E)E\subseteq\Psi(E), and Definition 2.1 now states this as a numbered definition; the submission stated it only in running text (the abstract, Section 1 and Section 2.2). This property makes novel success imply the original requirement, and it is used in the statement that Coredim\mathrm{Coredim} is either Cdim\mathrm{Cdim} or infinite (a finite safe zone is its own core).

If the restated results are now checkable, we would ask you to reconsider the assessment in that light.

Reviewer_MVU3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In language generation in the limit, a generator observes examples from an unknown language and must eventually output valid strings that have not yet appeared among the examples.

This paper studies a stronger novelty requirement: each output must lie outside the "span" of both the observed examples and the generator's own earlier outputs. The span of a set of strings EE consists of the strings that add nothing new once EE has been said, so it contains EE and grows with EE. For example, if strings are grouped into topics, the span of EE contains every string on a topic that EE already covers, and each output must be on a new topic. If strings are vectors, the span is the linear span, and each output must point in a new direction. If strings are logical formulas, the span is the set of logical consequences, and each output must say something that does not follow from what has been said. The first two examples are matroids, where the span of a set is its matroid closure and the size of a set is measured by its rank.

The authors introduce the notion of a "finite core": a set has a finite core if it lies in the span of finitely many of its own elements. In the classical setting, where the span of a set is the set itself, a set has a finite core exactly when it is finite. Their first result shows that, for every span, uniform generation is characterized by the closure dimension of Raman et al. with finite sets replaced by sets that have a finite core, and that this dimension either equals the classical closure dimension or is infinite. Their second result shows that if only the examples count toward the span, the classical characterizations of non-uniform generation and, for countable classes, generation in the limit also extend to every span in the same way. Then they study the case where the generator's own outputs also count, and their third result shows that these characterizations still hold under a technical condition on the span, which they call "local Noetherianity" and which every matroid satisfies. For spans without this condition, they give examples in which a single invalid output permanently prevents novelty. For instance, an output that contradicts a later example makes every statement follow from what has been said. Finally, they apply these results to matroids and to several matroid constraints at once, such as requiring each output to have both a new topic and a new style.

Critical strengths
  1. The model is a natural extension of language generation in the limit. It captures the distinction between producing different strings and producing different content. The results are also natural extensions of results obtained in earlier work on language generation in the limit (though not requiring more work).

  2. The proofs extend existing generation arguments naturally, and some of the extensions are technically interesting. In particular, the distinction between spanning a language from within and spanning it using earlier outputs outside seems interesting.

Critical weaknesses
  1. Many of the positive results closely parallel existing characterizations of language generation in the limit. Once sets with a finite core replace finite sets, several of these results are expected, and their proofs follow the classical ones closely, as the paper itself notes (e.g., Theorem 4.1 follows Raman et al., and Theorem 5.2(a) uses the critical-language construction of Kleinberg and Mullainathan). This limits the technical novelty of a substantial part of the paper.

  2. When the generator's own outputs count, the paper establishes characterizations of non-uniform generation and, for countable classes, generation in the limit under a local Noetherian assumption. The paper explains the mathematical role of this condition (Theorem 5.3), though its broader relevance beyond the matroid instances is less clear to me. In particular, logical consequence, one of the three motivating spans, is not locally Noetherian. In classical propositional logic, every set of formulas lies in the span of {p,¬p}\{p, \neg p\}, while the set of atoms has no finite core. Example 5.5 shows that, already for literals, generation in the limit can then be impossible. A full characterization would be interesting, and additional natural examples of locally Noetherian spans beyond matroids would help motivate the current focus.

  3. The exposition is generally understandable, although the writing style is unusual and some of the phrasing is awkward or overly compressed. The abstract and introduction rely on slogans such as "Finiteness was compactness" and "The uniform price is free or fatal."

Questions for authors

Please see weaknesses above.

2 comments

Response to Reviewer MVU3 (1/2)

Thank you for the summary and the logic example, which W2 below extends to a case without any contradiction.

W1 (novelty beyond the classical characterizations). Partly agreed: for Theorem 4.1, Proposition 5.1 and the sufficiency in Theorem 5.2(a) (the scaffold of Kleinberg and Mullainathan) the contribution is the notion of a finite core, under which the classical proofs go through, as Section 1 says. What does not come from those proofs:

  • Poisoning. When the generator's own outputs count, an output outside the target can, together with examples of the target, span the target. Example 5.5 is a countable class whose finite subclasses are all generatable from the first round, yet which is not generatable in the limit: the adversary simulates the generator and turns its first successful output into poison. For the identity span a generator's own outputs never hurt, so this has no counterpart under plain novelty (replay, where outputs re-enter as examples, is a different mechanism). Proposition 5.6 relabels this class into a second span that agrees with the first on every set inside a language, up to the relabelling, and gets the opposite answers.
  • Where the classical proof breaks. In the converse of Theorem 5.2(b), earlier outputs can lie outside the current safe zone, so the classical level-selection argument needs finite domination dimension, not finite core dimension. For a finitary closure operator, Theorem 5.3 shows that the domination form is exact, uniformly and in the limit, precisely when the span is locally Noetherian. Beyond such spans the characterizations stated with finite cores can fail (Example 5.5) or survive (Proposition F.6).
  • Several matroids at once. The union of the matroid spans need not be idempotent, so it is not a matroid, and a common independent set inside the span of EE can have k∣E∣k\vert E\vert elements rather than ∣E∣\vert E\vert; Proposition 7.1 replaces the rank bound by this capacity bound, from Steinitz exchange in each matroid, and Proposition 7.2 shows that infinite rank in each matroid is not enough. Theorem 6.2 is a simpler but exact criterion for when rank novelty and plain novelty coincide.

The two-language phenomenon of Theorem 6.1 has a counterpart in Li, Raman and Tewari (Example B.3), which the general response credits; where the submission's Section 6 said "no classical counterpart", it now says "no counterpart under plain novelty".

Response to Reviewer MVU3 (2/2)

W2 (local Noetherianity beyond matroids; logic). Agreed on the facts: classical consequence is not locally Noetherian, and with own outputs the non-uniform and limit levels beyond such spans are open.

As a theory of classes, a locally Noetherian span is no more general than a topic matroid (Proposition F.2, new); what is specific to a span is which of its sets have a finite core. Every span Ψ\Psi has a topic matroid MΨM_\Psi, with topics D0=Ψ(∅)D_0=\Psi(\emptyset) and Dn∖Dn−1D_n\setminus D_{n-1}, where DnD_n is the span of the first nn strings of the fixed enumeration. Its sets of finite rank are exactly the sets that finitely many strings span. So a locally Noetherian span generates the same classes as MΨM_\Psi: uniformly with the same threshold, non-uniformly, and in the limit for countable classes. For a finitary closure operator, local Noetherianity is exactly the condition under which this holds for every class (Proposition F.2(iii), via Theorem 5.3). The reduction does not compute anything by itself: MΨM_\Psi is defined from an enumeration, and to apply the theorems to a given span one must know which of its sets have a finite core. That is the span-level work. Proposition 7.1 does it for several matroids at once (no infinite common independent set), and the new Appendix F.4 does it for consequence confined to topics over finitely many atoms each (Proposition F.3) and for integer spans (Example F.5), where a set has a finite core iff its rational span has finite dimension; the integer span is not a matroid, as 2e12e_1 lies in the span of e1e_1 but not conversely. More generally, submodule spans over a Noetherian ring are locally Noetherian and, as over the integers, need not be matroids.

For consequence, the positive results are uniform generation for every class of finite core dimension (Theorem 4.1), every level when novelty is measured against the examples alone (Proposition 5.1; the limit for countable classes), and every level with own outputs when consequence is confined to topics (Proposition F.3). With own outputs and unrestricted consequence, the non-uniform and limit levels can fail through explosion (Example 5.5), and your example extends to failure without any contradiction (Remark F.4). Take K0={pj:j≥1}K_0=\lbrace p_j: j\geq1\rbrace and, for t<jt\lt j, Lt,j={p1,…,pt}∪{pj∨qj,i:i≥1}L_{t,j}=\lbrace p_1,\dots,p_t\rbrace\cup\lbrace p_j\vee q_{j,i}: i\geq1\rbrace. No language has a finite core, so the class is generatable in the limit when only the examples count. With own outputs it is not: the adversary simulates the generator on p1,p2,…p_1,p_2,\dots, finds its first success pjp_j at some round tt, and commits to Lt,jL_{t,j}, enumerated starting with p1,…,ptp_1,\dots,p_t; from round t+1t+1 the consistent set {p1,…,pt,pj}\lbrace p_1,\dots,p_t,p_j\rbrace is in the history and spans Lt,jL_{t,j}. A characterization beyond local Noetherianity remains our first open problem; Proposition 5.6 shows that it must look at sets that straddle languages.

W3 (style). Agreed. Beyond the abstract, we rewrote the paragraph of Section 1 on the generator's own outputs, the passages of Section 5 on poisoning, the domination form and the master keys, and single sentences in Sections 1-4 and 6-8, removing "Finiteness was compactness", "free or fatal" and the slogan headings of Section 1; the title is unchanged. Sections 1-2 now rest on a boxed list of the main results and numbered definitions.

If these points answer the question of what is new, we would ask you to reconsider the assessment.

Comments on the paper

2 comments

General Response (1/2)

We thank the three reviewers. No reviewer identifies a specific mathematical error; Reviewer HgEx reports that the writing prevented checking the proofs. The concerns are presentation (Reviewer HgEx and Reviewer MVU3 in their text, Reviewer f3Zs in the presentation rating), what the paper adds to prior work (Reviewer MVU3, Reviewer f3Zs), the reach of the locally Noetherian condition (Reviewer MVU3), and concrete instances (Reviewer f3Zs).

1. Presentation (Reviewer HgEx, Reviewer MVU3, Reviewer f3Zs). We agree that the abstract, Section 1 and Section 2 made the definitions and theorems hard to extract, and that the slogans got in the way. The revised PDF rewrites the abstract sentences Reviewer HgEx quotes, the paragraph of Section 1 on the generator's own outputs and the passages of Section 5 on poisoning, removes the slogans from the text (the title is unchanged), adds a boxed list of the four main results to Section 1, and states span, finite core and local Noetherianity as numbered definitions (Definitions 2.1, 2.5). The rewrite also corrects a statement of the submitted abstract: poisoning is not impossible under local Noetherianity; it can occur only for targets with a finite core, which are lost anyway, so it does no harm (Theorem 5.2). The reply to Reviewer HgEx gives the model and the main results in self-contained form.

2. What the paper adds (Reviewer MVU3, Reviewer f3Zs). Each output must lie outside the span of everything said so far, the generator's own outputs included.

  • With "finite" read as "has a finite core" (spanned by finitely many of its own strings), the characterizations of Kleinberg and Mullainathan and of Raman, Li and Tewari hold uniformly for every span (Theorem 4.1) and at every level when only the examples count (Proposition 5.1; the limit for countable classes); the core dimension equals the closure dimension or is infinite. The proofs are the classical ones read through cores; the contribution there is the notion.
  • Counting the generator's own outputs creates poisoning: an output outside the target can, together with examples of the target, span the target. Example 5.5 is a countable class whose finite subclasses are all generatable from the first round, yet which is not generatable in the limit; Proposition 5.6 relabels it into a second span that agrees with the first on every set inside a language, up to the relabelling, and gets the opposite answers. Replay (Racca, Valko and Sanyal, 2026), where earlier outputs re-enter as examples, is a different mechanism; there too the uniform level is unaffected and the non-uniform level is not.
  • For a finitary closure operator, local Noetherianity is exactly the condition under which the span generates the same classes as a topic matroid built from it (Theorem 5.3, Proposition F.2). Under it poisoning does no harm; beyond it, poisoning can be fatal.
  • For matroids: an exact criterion for when rank novelty and plain novelty coincide (Theorem 6.2), and several matroids at once, where infinite rank in each is not enough (Propositions 7.1-7.2).

Li, Raman and Tewari (metric spaces) is the closest prior work (details in the reply to Reviewer f3Zs). Their ball around each example is an examples-only span, with covering numbers rather than counts measuring the sample. Their current version (v2, October 2026) measures safe zones by covers centred inside them, which for the ball span is a finite core. Our paraphrase trap is an instance of their model (distance 1 between paraphrases and 2 otherwise, with radii εa<εg=1\varepsilon_a\lt\varepsilon_g=1), and their Example B.3 shows the same phenomenon; the submission did not cite that example, and the revision cites it in Sections 1 and 6 and in Appendix H.

3. Locally Noetherian spans (Reviewer MVU3). Classical consequence is not locally Noetherian, and with own outputs the non-uniform and limit levels beyond such spans remain open. The condition cannot simply be dropped (Example 5.5; Remark F.4 gives a fatal case without any contradiction). The characterizations stated with finite cores can still hold beyond it (Proposition F.6). The reply to Reviewer MVU3 lists locally Noetherian spans that are not matroids, such as integer spans (Example F.5); there local Noetherianity is the Noetherian condition of algebra rather than exchange, and the span-specific work is deciding which sets have a finite core.

Changes in the revised PDF, all in blue; the main text ends on page 10:

General Response (2/2)

  • Abstract: the sentences Reviewer HgEx quotes, rewritten. Section 1: a boxed list of the main results, the paragraph on own outputs rewritten, and a paragraph comparing with Li, Raman and Tewari.
  • Section 2: Definition 2.1 (span) and Definition 2.5 (finite core, finitely dominated set, locally Noetherian span).
  • Sections 3-8: plain sentences instead of slogans; Section 6 credits Li, Raman and Tewari's Example B.3; Table 2 (page 10) gives four worked instances (Reviewer f3Zs).
  • Appendix F.4: Propositions F.2-F.3, Remark F.4 and Example F.5, with proofs; Section 5 points to Proposition F.2 and Remark F.4.
  • Appendix H: the metric comparison updated to their v2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.