Differential Feature Upsampling

We present DiffUp, a novel feature upsampler that exploits differential kernels to enhance details. Since feature upsamplers evolve from hand-crafted to content-aware designs, they excel at detail preservation yet still produce ambiguous boundaries. In this work, we uncover a boundary suppression phenomenon behind the de facto use of softmax kernel normalization: positive-only weights reduce boundary sharpness. Inspired by zero‑mean filters, we introduce signed weights and differential kernels to amplify the differences. However, introducing signed weights alone cannot guarantee the amplification behavior; their spatial distribution also matters. We show that the same differential formulation can enhance details while also executing local averaging. We therefore establish conditional guarantees for when and where the behaviors can take place. By embedding our insights into DiffUp, we show through controlled experiments and four dense prediction tasks that DiffUp outperforms or matches state-of-the-art dynamic upsamplers. Notably, it improves the boundary IoU on COCONut by up to 4.37% over bilinear interpolation. Our approach suggests yet another design principle for modern feature upsamplers, i.e., amplification over mere preservation.