How Large a Break Can Hide in a Scaling Law?

Scaling laws guide extrapolation from a small set of training runs, but a smooth fit does not establish that the exponent is stable. We study the detection of a continuous change in exponent when the breakpoint is unknown. For designs with 4–20 model sizes, comparing a scanned-breakpoint statistic with a fixed-break F distribution gives false-positive rates of 18.1–32.0% at a nominal 5% level. Simulating the complete scan under the null restores rates to 5.0–5.3%. We characterize the test’s invariance to the baseline exponent and noise scale, and quantify detection power over sample size, noise, and break location. Across two decades of model size, designs with 6–8 observations and 2–5% relative noise require exponent changes of 0.1–0.4 on the tested grid to reach 80% power. Power decreases when the break moves toward either end of the range. A six-model Pythia case study illustrates a separate issue: adding a loss floor reduces log-scale residual sum of squares by a factor of 53, showing how smooth curvature can produce a strong apparent break under a pure power law. We provide a reproducible calibration and power-analysis procedure, and recommend reporting detectable changes alongside fitted exponents.