Making Depth Matter: Complementary Representation Learning for Deep Recommender Systems

Making a recommender deeper does not necessarily make it better. We study this problem through the joint design of representation learning and reuse, introducing DeCORA (Depthwise Correlation-Regularized Aggregation), a wrapper that couples token-conditional cross-depth correlation control with depth-prefix aggregation. A training-only objective regularizes source correlations with learned, normalized pair weights, while depth-specific maps shared across sources aggregate the embedding and all available hidden representations, retaining the backbone's interaction operators. Our conditional analysis connects correlation control to coordinate-function overlap and its transport through affine maps, clarifying the roles of target alignment and access after aggregation in making distinct responses useful for prediction. Across four structurally different backbones on Avazu and Criteo, DeCORA improves AUC and Logloss over vanilla in all 40 matched Base–$8\times$ configurations, with a mean absolute AUC gain of $0.00156$ at $8\times$ depth. Among the component and parameter-matched variants, only the full method improves mean AUC in all eight backbone–dataset pairs from $8\times$ to $16\times$; representation diagnostics further show that the full method consistently exhibits the geometric patterns motivated by our analysis. The code is available in an anonymous repository.