Density Is Not Training Utility: A Learning-Response Analysis of Synthetic Data

Visually realistic synthetic images do not necessarily improve downstream performance, and evaluating each synthetic dataset through retraining is costly. We study this problem through the *learning response*: how training on synthetic samples changes predictions on real target samples. Our analysis relates utility to whether these changes correct existing target errors and to the curvature of the target loss. We show that utility depends on current data coverage, and that image distributions alone cannot determine utility rankings across tasks. We propose Interventional Godambe Risk Tomography (IGRT) to predict changes in target negative log-likelihood without retraining for each candidate set. Using a reference model fitted on real data, IGRT combines local approximations of parameter shifts with uncertainty in fitted parameters due to data sampling. Controlled experiments show that its predictions capture the effects of candidate labels, current data coverage, and augmentation size. Across multiple downstream tasks, IGRT predicts the direction and magnitude of augmentation effects and selects subsets that improve performance over distribution-based selection strategies and model-based data valuation methods under matched data and training budgets.