When Is Gradient-Based Data Selection a Property of the Predictor?

Gradient-based data selection is commonly interpreted as identifying examples that are informative for a given predictor, yet neural-network parameters are not unique representations of the predictor. We study predictor-level well-definedness of the resulting discrete selection under function-preserving reparameterizations. Across BADGE, CRAIG, and GradMatch, equivalent predictors replace $18.8$–$63.0\%$ of a query batch while their logits remain unchanged to numerical precision; in end-to-end LoRA fine-tuning, factor-coordinate selection replaces $30.4$–$43.7\%$ of MiniLM queries and, in the analytical factor-basis comparison, $70.3\%$ on average for ModernBERT. Per-example gradients transform as covectors, so a fixed parameter-space metric generally generally changes the geometry seen by the selector. We characterize when Gram geometry is preserved exactly or up to scale, and give selector-specific constructions in which the resulting distortion crosses a discrete selection boundary. For Fisher-based selection, we further show that a coordinate-fixed prior can break invariance even when the Fisher terms transform covariantly. Finally, repeated acquisition reveals that query non-identifiability and downstream utility are distinct: acquisition trajectories can diverge strongly while terminal accuracy remains similar in the settings studied. These results identify parameterization as a source of data-selection non-identifiability that is separate from conventional questions of subset utility.