Learning The Vocabulary of Mathematics From Data

Symbolic regression (SR) searches for formulas built from a set of functions fixed before the search, which we call its vocabulary. When a law needs a function outside that set, the search can only return a longer expression that approximates the missing function with the functions it has. Yet one rarely knows in advance which functions a law will need. We show that the vocabulary of mathematics can be learned from data. We add to an SR library one learnable operator, a spline or a small neural network that the search can insert anywhere a built-in function can go. All occurrences of the operator share one parameter vector, which is fitted together with the numeric constants. We tested the search on three laws that contain Bessel's $J_0$, with only elementary functions in the library. The learned operator matched $J_0$ in $13$ of $20$ runs. In most of these runs the three selected formulas had $12$ nodes in total, the same as a search whose library contains $J_0$. Searching with only the elementary functions, even when given four times as much time, returned formulas of about $50$ nodes with higher error. The runs whose operator matched $J_0$ were exactly those with the lowest training loss. We prove that several laws sharing one operator can determine it, up to the scale and offset of its output, even when no single law does. It is enough that the laws together rule out every change of the operator's shape, and that no other set of formulas of the same or smaller size can reproduce how the laws change with their parameters. To test the case in which a needed function is missing, we also delete familiar functions from the library and check whether the operator recovers them. It recovers the square root in about two thirds of runs, and addition in $21$ of $50$ runs when the operator is a neural network. On recordings of a real pendulum, three of ten runs select the energy law with a cosine-shaped operator. Functions such as $\sqrt{\cdot }$ and $\sin$ became standard because each describes many different laws compactly, and a search that fits new functions from data could find useful functions that do not yet have names.