TameProbe: An Empirical Tameness Spectrum for Neural Decision Boundaries

Model-theoretic o-minimality (“tameness”) has recently been proposed as a lens on neural network geometry, but it is a binary structural property that is not directly measurable on a trained model. We turn tameness into an operational, continuous diagnostic spectrum: two cheap, model-agnostic statistics of a network's decision boundary—the oscillation complexity ($\mathrm{OC}$, sign flips and local extrema of the decision margin along random 1D probes) and the decomposition complexity ($\mathrm{DC}$, number of distinct linear regions crossed). Across a controlled matrix of MLP architectures, seeds and post-training operators (an alignment-style Jacobian/weight regularizer, knowledge distillation, and 8-bit quantization; $14,400$ measured slices), we find that (i) tameness is not scalar: the alignment proxy lowers both $\mathrm{OC}$ ($-35.4\%$) and $\mathrm{DC}$ ($-30.3\%$), whereas distillation lowers $\mathrm{OC}$ ($-17.9\%$) while raising $\mathrm{DC}$ ($+1.0\%$), so the two axes decouple; (ii) 8-bit quantization does not move the spectrum ($p>0.18$), a negative result that validates the metrics as capturing function geometry rather than numerical noise; and (iii) on an external adversarial-robustness anchor ($n=120$), $\mathrm{OC}$ strongly predicts PGD robustness (Spearman $\rho =-0.68$, $p<10^{-17}$; $\sim 3\times$ the effect of $\mathrm{DC}$), connecting our spectrum to the curvature–robustness literature. We further show, on a balanced compositional truth concept across five same-family base/instruct LLM pairs, that both base and post-trained LLMs already sit at the tame end (a nonlinear RBF kernel never beats a linear probe), with post-training leaving this position essentially unchanged. TameProbe gives a reproducible, quantitative vocabulary for how far along the tameness spectrum a model sits and how training operators move it.