Failure Is Not Absence: Selective Low Rank Supervision For Dense Retrieval

Dense retrieval failures are often attributed to representations that do not sufficiently capture the discriminative structure required to distinguish relevant from non relevant documents. We challenge this view by showing that retrieval failure does not necessarily imply the absence of such structure. Using frozen single vector representations, we show that queries that fail under the original retrieval function are recovered through low rank readouts. We further find that the discriminative signals enabling this recovery are extracted even from very low dimensional subspaces. By decomposing the full space score into selected subspace and residual contributions, we show that the low rank contributions associated with recovery are counteracted by residual contributions, leading to dense retrieval failure. Building on these findings, we propose Selective Low Rank Supervision, which selects beneficial rank one signals and aligns the full embedding representation with the selected signals during training. The proposed method requires no additional subspace computation at inference, while outperforming standard contrastive fine tuning. These results indicate that the low rank structure revealed by our analysis is not only informative for understanding retrieval failure, but can also provide an effective training signal for improving the full embedding representation for retrieval.