Jacobian Rank Collapse in Decision-Focused Learning

Decision-focused learning (DFL) trains predictors through downstream objectives, yet changing the loss does not necessarily create a new direction for updating the model. We study this limitation through the predictor Jacobian. At rank one, all nonzero backpropagated loss gradients are collinear; a spectral bound characterizes near-collinearity when a leading singular direction dominates and the gradients have nonzero components along it. This local restriction does not imply that different losses share an optimum. We test its empirical relevance in sparse index tracking across six equity markets, and examine transfer to weather, air quality, and shortest paths. Across 38 one-parameter equity configurations spanning a 54× range of sparsity, DFL gains over MSE remain below 1.8% (median 0.25%), with no detected monotone association with sparsity (ρ = −0.16, p = 0.35). Structured predictors yield larger gains in several markets and up to 9.4% on air quality. A calibrated effective-dimension proxy organizes these outcomes: the archived blind-evaluation record contains 27 correct predictions among 31 non-abstaining cases, with two additional abstentions. Transfer results also expose the limits of fixed thresholds. For higher-capacity models, covariance regularization mitigates task-training instability. Together, these findings motivate assessing the gradient directions a predictor can express, validating the diagnostic in the target domain, and regularizing task training when needed.