Beyond Fixed Precision: Measuring and Tracking Minimum Sufficient Precision in Learning

Mixed-precision training typically assigns one of a few hardware-supported formats to each layer, operator, or graph region. Such assignment rules choose among available formats without measuring the precision that a given learning computation actually requires. We introduce Minimum Sufficient Precision (MSP), the lowest precision that preserves a training step's numerical or learning-level behavior for one operator execution. We measure MSP with controlled precision sweeps across four model families and a controlled stress workload, relate its variation to numerical and learning-side factors, and use it to provision executable precision levels during training. In 56% of 120 valid comparisons the numerical and learning-level criteria select different MSPs, so numerical accuracy alone does not guarantee that training behavior is preserved; the requirement also moves with training state, and refreshing more often does not consistently improve fidelity. Precision is therefore a local, criterion-specific, and time-varying computational requirement to be measured and tracked, rather than a fixed datatype assignment.