Beyond Static Embeddings: Finite-Query Certificates for Local Representation Stability

The geometry of finitely observed embeddings does not determine the local stability of the representation map. We investigate this information gap using a finite-query framework. Within the class of continuously differentiable functions, identical activation observations can correspond to arbitrarily large local sensitivity. Moreover, perturbation queries can leave directions outside their span unidentified. Using a classical anchored interpolation operator, we can recover the Jacobian restricted to the covered subspace, accounting for a curvature-conditioning error that relies on an independently provided local smoothness bound. This analysis yields two-sided sensitivity estimates, finite-scale displacement bounds, and a prediction-invariance condition that accounts for the readout and decision margin. We present a 1D construction that meets the recovery bound for a one-sided endpoint query. This operator analysis leads to Secant-Frame Regularization (SFR), an input-Jacobian-free objective that targets the largest reconstructed logit response. Through experiments, we instantiate the conditional bounds. On the CIFAR-10 dataset, SFR demonstrates improved robustness in evaluated covered transformations compared to matched-query pairwise consistency. Our findings link finite-representation observations to a defined local stability analysis and distinguish conditional certification from empirical regularization.