acceptodds
Under review as a conference paper at ICLR 2027

Why are numeral systems regular? The role of learnability and generalisability

Abstract

Human recursive numeral systems (e.g. counting systems such as Mandarin or English base-10 numerals) are, like many other linguistic systems, highly regular, i.e. they can be represented by a simple grammar. We are interested in modelling what pressures select for such representations over the course of linguistic evolution. Following prior work that relates cross-linguistic tendencies to biases in learning, we hypothesise that regular systems are common because regularity allows the extraction of highly-learnable, highly-compressible representations. We compile a dataset of 20,732 unique numeral systems (both human and artificial) and experiment with both reinforcement and supervised learning as high-level analogues for human learning. Experiments confirm that highly regular human(-like) systems are indeed more learnable than unattested irregular systems, but only when learnability is taken as the capacity to derive a generalisable representation of the whole numeral system from skewed, limited data. Intriguingly, our results indicate that the advantage of regularity is more significant for reinforcement learning. We also conduct exploratory analyses which show that a multitude of factors affect numeral systems' learnability, but crucially, regularity robustly remains one of the key predictors. Our results contribute to the body of work linking learnability to cross-linguistic prevalence.

Primary area:
applications to neuroscience & cognitive science
Keywords:
learnability, regularity, generalisability, skewed input, constrained variation, computational cognitive science, numeral systems, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_gEGgOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Many natural language number systems are regular (e.g. 78 is pronounced seven + ty + eight, i.e. 7 * 10 + 8). This paper hypothesizes that this is because such systems are easier to learn. It tests this hypothesis using LSTMs trained either in supervised learning or reinforcement learning, and in general finds support for the hypothesis. Another very important predictor is whether the number system follows PEMDAS (the usual order of precedence of parentheses, multiplication, addition).

Critical strengths
  1. The paper is very well written and provides an excellent overview of the topic.
  2. The experiments and their statistical analysis are thorough.
Critical weaknesses

Some of the findings do not necessarily require an experiment; for example, it is fairly obvious that to find effects of regularity you want the distribution of the test set to be uniform rather than identical to the training distribution (power law), so it seems like you could just skip the IID experiment. But more fundamentally, I find it hard to imagine a case where regularity wouldn't affect learnability; of course if all of the numbers in English' are regular but 9338 is pronounced "avocado" it will be very hard to learn that 9338 is pronounced "avocado". I don't think the paper does the most compelling job of explaining what specific outcomes of the experiment would be empirically illuminating (which non-obvious possibilities are ruled out). To be fair, this is something that may be resolvable during the discussion period, and I'd be happy to engage with the authors on this.

Questions for authors
  1. I don't really know what to make of the cognitive plausibility of the particular modeling choices (LSTM, RL vs. SL). Are some of these more like what we imagine humans to be? Is there really such thing as "learnability" that's not relativized to a particular learner?

  2. I got a little lost in the discussion of PEMDAS, which seems particularly important in the statistical model. Do we imagine humans and LSTMs as having an inductive bias that favors number systems that follow PEMDAS?

1 comment

We thank the reviewer for their comments and we appreciate their compliments for our presentation and thoroughness of experiments and analysis.

Critical Weakness 1: To address your comments (and a similar comment from another reviewer), we agree that "regularity facilitates learnability" is not particularly a novel insight. However, we would like to point out that our work adds important nuances to this claim. First, in our view, the lack of learnability advantage in Experiment 1 (especially for RL) was not at all obvious a priori--perhaps a small exposure to rare numerals would have sufficed for a regularity advantage. Only by doing both Experiment 1 and 2 did we learn that the capacity to generalise to rare numerals is key. The non-linear shape of the learnability curve in Experiment 2 (and 3) also suggests that regularity has little to no effect on learnability once the system becomes sufficiently irregular; in our view, this is also not obvious. Second, our work highlights a crucial distinction (often conflated in prior work on language efficiency analysis) between frequency of use and communicative need, the latter of which actually shapes linguistic systems. In the case of recursive numerals, the use of numerals is highly skewed, and yet we place a lot of importance in expressing rare numerals precisely. This distinction may also explain systems in other domains where a bias for certain meanings has been found, such as obligation modals in modal verbs (Uegaki et al., Deontic priority in the lexicalization of impossibility modals, 2023). Also, our multivariate regression modelling shows, non-trivially, that regularity remains important and key even when incorporating other possible factors. Finally, the new main section we have added in response to reviewers (Experiment 3, page 9) reveals that among human and human-like systems, irregularity does not reduce learnability over the whole system (ρ=0.27\rho = 0.27) but does reduce generalisation to unseen numerals (ρ=−0.51\rho = -0.51) (both Spearman ρ\rhos, LSTM under RL), corroborating the connection between regularity and learnability (as generalisability).

To answer your specific questions:

Q1: We agree with the reviewer that learnability is learner-relative. Both LSTMs and transformers (which we have added experiments for in the main paper, see Experiment 3, page 9) are high-level analogues for human learning--at least to the extent that all algorithms learning the same target system exploit data similarly; we remain agnostic as to whether they are truly realistic cognitive models. That said, between the two training regimes, our results tentatively point to RL as the closer analogue in this setting: it exploits structure in the data--something we know that humans do--whereas SL also learns highly idiosyncratic, unattested systems efficiently (§4.2). Comparing the two architectures, we can safely say that the overall trend and separation between regular and irregular systems hold for both (Spearman ρ=0.84\rho=0.84 under RL and 0.940.94 under SL across systems), but the two agree only partly on the ordering of individual human(-like) systems (ρ≈0.5\rho\approx0.5, cf. Appendix J). Our claims are therefore about general learnability trends.

Q2: We actually view the PEMDAS bias primarily as a property of the data. We made the operational assumption that the space of possible numeral grammars will be those generating numerals that will almost always evaluate to the correct numbers if one follows PEMDAS (Hurford, The linguistic theory of numerals, 1975), and deviations from PEMDAS are unsystematic. The systems with consistent evaluation patterns are those whose correct way of evaluating numerals consistently conform to PEMDAS, resulting in an apparent "PEMDAS bias". Humans and models alike can take advantage of the consistency of evaluation patterns (which happens to be PEMDAS-conforming in this case), e.g., through overhypothesis learning, and thus learn PEMDAS-conforming systems more easily.

Reviewer_HEWROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper tries to uncover the motivation behind why regular numeral systems were built and utilized in human languages. To do so, the authors utilize deep learning models to understand what systems are easier to learn and generalize from. Their primary finding is that regularity improves learnability and generalizability, alongside a few other factors.

Critical strengths
  1. I like this work because it is notably different from works generally submitted to such conferences - the authors are making use of deep learning training to understand why recursive numeral systems used by humans are regular.

  2. Some of their findings fit well with existing works in literature which show that RL learns general patterns while supervised training often memorizes.

Critical weaknesses
  1. Generalization has not been assessed well enough. If RL were truly drawing general patterns, wouldn't it make sense to test generalization to unseen or held out numbers? It seems as though the current generalization test is only on less frequently observed numbers. Curious to hear the authors‘ thoughts on this.

  2. I'm not sure the work presents novel enough insights. "More regular systems with learnable patterns are easier to generalize from" - this is intuitive and well known. Additionally, past works have shown that human languages exhibit systematic regularities. I would like to ask the authors - what new understanding does this work provide that has not been established before? Or is it the "deep learning to understand how language evolved" angle that is their core contribution?

Questions for authors

I have listed my concerns and questions above.

3 comments

We thank the reviewer for their comments and for engaging with our paper, and for recognising the originality of our work. To address their questions:

Critical weakness 1: We agree with the reviewer that it is worth testing generalisation more thoroughly, to see if our artificial learners are indeed learning general patterns. Following their advice (and a similar comment from another reviewer), we therefore added a new experiment to the main paper (Experiment 3, page 9), run on the 3,625 systems of our regression sample, which covers the space of systems evenly (we are happy to extend it to all 20,732 systems for the final version), withholding a random 10% and 20% of the infrequent numerals. The results clearly show that generalisability is very much connected to regularity even when numerals are withheld from training, and that learnability over unseen numerals is especially good at separating highly regular and irregular systems: median learnability on unseen numerals is 7,234 for human systems, 6,842 for human-like and only 407 for artificial ones (LSTM under RL, out of a maximum of 30,000). The models do indeed learn to generalise to unseen numerals within their training range, but crucially almost only when learning highly regular systems from skewed, scarce data.

Critical weakness 2: We agree with the reviewer that "regularity facilitates generalisability" is not particularly novel. We view our work as adding nuances to this claim, such as the fact that the learnability advantage is not linear, and that regularity remains an important predictor even when other factors are incorporated (as in our regression model). Importantly, we would like to highlight the following. First, it was not obvious to us that training learners on a skewed distribution would not yield a learnability advantage as in Experiment 1--perhaps small exposure to rare numerals is enough to drive learnability. Our results show that, at least in our setups, regularity yields little to no advantage unless we look at rarer numerals. This suggests that the skewed distribution would likely not have been the main driver of the development of recursive numerals (and other compositional systems). Crucially, it also highlights that assuming the distribution of the frequency of use to be the same as the communicative need distribution (an unchallenged assumption made in much previous work on efficiency analysis of language) is not always apt. This could also extend to other domains where a bias (not driven by frequency) for certain meanings has been shown, e.g., for modal meanings where obligation modal verbs are possibly considered more important (Uegaki et al., Deontic priority in the lexicalization of impossibility modals, 2023). Finally, the new main section reveals that among human and human-like systems, irregularity does not reduce learnability over the whole system (ρ=0.27\rho = 0.27) but does reduce generalisation to unseen numerals (ρ=−0.51\rho = -0.51) (both Spearman ρ\rhos, LSTM under RL).

I thank the authors for their comments and additional experiments. They do help. However, I'm still not convinced this paper carries the kind of new insights that is generally required for acceptance at such a conference. To be clear, I think this work is different and nice, but lacks the novelty in its insights that would make me vote for acceptance. There is also the concern of how much utility such a piece of literature has towards pushing the field of machine learning that is often considered at these conferences - but this wasn't the focus of the authors to begin with so I cannot judge through this lens too harshly but it is still a concern. I will keep my original rating with low confidence.

We appreciate and respect the reviewer’s response. We nevertheless feel that our paper would still be a great fit for the conference. We believe that, as indicated by our chosen primary area (applications to neuroscience & cognitive science), this paper represents a significant application of machine learning to the field of cognitive science.

Reviewer_kRcGOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks the question of whether regularity in numerical systems influences the learnability of those systems positively. For this, the authors use a large collection of numerical systems, both human and artificial, to test how quickly and easily LSTM models can learn the underlying numeral-to-numeric mappings, using both reinforcement and supervised learning. The authors find that learnability correlates with regularity in this setup, but mainly when training on skewed distributions and testing uniformly across numbers, giving more importance to infrequently encountered numerals.

Critical strengths
  • The paper explores an interesting connection between regularity, learnability, and linguistic evolution, providing useful insights into why numeral systems might have evolved to be so regular.

  • The authors consider a large collection of human and artificial numeral systems and evaluate different training and testing distributions, which makes the analysis more informative.

Critical weaknesses
  • The authors consider only LSTMs as learners. It is unclear whether the conclusions generalize to other architectures or learning approaches, e.g., inductive program learning. A regularity advantage in LSTMs might reflect properties of the architecture rather than a general property of learnability. It is therefore unclear to me how generalizable the results are to learnability in general.

  • The evaluation does not really establish whether the models learn the underlying compositional rules that would also generalize to unseen numbers. Since training and testing cover the same numerical range, models might benefit from repeatedly seeing rare numerals rather than learning general rules. While this does not invalidate the learnability results, it remains unclear to what extent the models actually learn compositional rules, which play an important role in how humans learn and produce numeral expressions (Holt & Barner, 2025).

Questions for authors
  • Q1: Have the authors considered testing generalization to larger numerical ranges, e.g., training on numbers up to 99 and testing on thousands? This could help understand whether the models actually learn recursive rules rather than just mappings within a fixed range.
  • Q2: Did the authors consider the frequency of individual morphemes as a potential confounding factor? Some systems might reuse certain morphemes much more frequently than others, making them easier to learn independently of regularity.
3 comments

We thank the reviewer for their comments and for engaging with our paper.

Critical weakness 1: We have added experiments to the main paper with transformer models (Experiment 3, page 9), the results of which confirm that the advantage of regularity for learnability is visible for both classes of models. Regarding other classes of learners like program induction learners, these would choose regular systems by construction (as their grammar is simpler/more compressible). This is why we chose a minimal learner with no built-in simplicity bias.

Critical weakness 2: We agree with the reviewer (and a similar comment from another reviewer) that testing our models' generalisation to unseen numerals would strengthen our results. We therefore added a new experiment to the main paper (Experiment 3, page 9), run on the 3,625 systems of our regression sample, which covers the space of systems evenly (we are happy to extend it to all 20,732 systems for the final version), withholding a random 10% and 20% of the infrequent numerals. The results clearly show that generalisability is very much connected to regularity even when numerals are withheld from training, and that learnability over unseen numerals is especially good at separating highly regular and irregular systems: median learnability on unseen numerals is 7,234 for human systems, 6,842 for human-like and only 407 for artificial ones (LSTM under RL, out of a maximum of 30,000). The models do indeed learn to generalise to unseen numerals within their training range, but crucially almost only when learning highly regular systems from skewed, scarce data.

In response to your specific questions:

Q1: We have partially addressed this question above -- regularity does indeed facilitate generalisation to withheld numerals within the 1-99 range. We are unable to implement exactly your suggestion in this question: the systems in our dataset are only defined for 1–99, so we cannot test on larger numbers (i.e. we do not have data for all languages on how those numbers are expressed). As the closest alternative, we attempted an experiment (not shown in the paper) where we withheld the top of the range (either 91–99, or 81–99) from training then tested generalisation to those numbers, i.e. testing generalisation beyond the trained range, rather than within it. We found that the models partially learned the relative magnitude of the numerals (e.g., when learning Spanish numerals (LSTM under RL): a model predicted 86 for 91 and 92, 87 for 93 and 94, 88 for 95, 89 for 96, etc.), possibly reflecting compression towards the trained range. This failure to extrapolate to numbers beyond the trained range has been shown elsewhere (e.g. Trask et al., Neural Arithmetic Logic Units, 2018; Wallace et al., Do NLP Models Know Numbers? Probing Numeracy in Embeddings, 2019). In other words, given highly regular systems, the models are able to generalise to specific, unseen numerals within a certain range (e.g., they may be able to learn 85 and 87 if 81, 82, 83, 84, 86, 88, 89 are in the training data), but not entire ranges (like 91-99 or 81-99).

Q2: It is unclear to us that the frequency of individual morphemes would make learning numeral systems easier (or harder). It is possible that having a highly skewed frequency distribution over individual morphemes hinders learning as learning all numerals requires learning all the unique morphemes, and a highly skewed distribution will hinder learning the rare morphemes necessary for constructing certain numerals. In our view, it is by forming regular forms that the reuse of individual morphemes can make a system more learnable, and thus we do not view regularity and frequency of individual morphemes as confounding. For instance, a regular system might use 5+15+1 to express the quantity 66 across numerals (e.g., 5+15+1 for 6, 10+5+110+5+1 for 16, 2∗10+5+12*10+5+1 for 26, and so on), whereas an irregular one might not consistently use 5+15+1 for the quantity 66 (e.g., 5+15+1 for 6 but 10+610+6 for 16). Crucially, however, regularity captures properties beyond the frequency distribution of individual morphemes (e.g., whether only 5+15+1 is consistently used for the quantity 66, and not an alternatively ordered form like 1+51+5). Moreover, the average frequency of a system’s morphemes in training is fully determined by its average numeral length and lexicon size, both of which are already predictors in our regression (§4.3), where regularity remains a key predictor.

I thank the authors for their detailed answer. I appreciate the added experiments on the transformer architecture and the held-out numerals, and I think they improve the paper. The finding that the models are not able to generalize to out-of-domain numerical ranges is interesting. I think it would be valuable to mention it in the discussion and conclusion.

Btw, I appreciate the update to the paper with regard to experiment 3, but I hope that the wording will be updated in case of a camera-ready version (motivate the experiments not only based on what the reviewers asked).

In regard of the changes, I'm raising my score to 3 (weak accept).

We thank the reviewer for their positive comments. For the camera-ready version, we would be happy to rearrange the paper so that the motivations for the new experiments follow more appropriately from our previous ones. This will entail running the new experiments on the extended dataset. We are also considering performing a similar regression analysis on the new results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.