TailBudget: When Is Prefix Truncation Worth Its Cost?

Randomly retaining an answer prefix can reduce computation without changing the expected supervised gradient. The harder question is whether this is preferable to keeping complete answers and tuning the batch size. We study this question through TailBudget, a finite-budget decision rule for frozen-gradient estimation. Starting from classical inverse-probability weighting, we charge the cost of learning whether a tail is inexpensive to omit. The resulting acceptance condition requires gross variance–cost savings to exceed the diagnostic budget fraction. A four-corner certificate handles uncertain gradient moments and execution costs; an integer allocation rule makes the comparison executable under an expected-cost budget. The guarantee applies to accepted policies, not to a cost-free fallback after diagnostics have been paid. Exact finite-vector checks and source-informed synthetic scenarios expose overhead reversals, batch-baseline confounding, and the limits of length-only decisions. Scenario scores are authored rather than checkpoint measurements or fitted estimates; previously reported CPU runs were unavailable for verification. The result is a precise account of when truncation earns its additional noise, rather than a claim of faster language-model training.