acceptodds
Under review as a conference paper at ICLR 2027

Not Just a Matter of Scale: Recovering Forensic Traces to Build a Forensic-Enhanced Vision Backbone

Abstract

Foundation models expose useful but uneven forensic evidence. We study whether a compact DINOv3-L backbone can recover evidence absent from its CLS readout, using DINOv3-7B only as a training-time teacher. Rank–nullity and orthogonal projection characterize this bottleneck and show when student patch features can reduce reconstruction error. We propose RELIC, which uses frozen teacher-relative residuals to guide persistent collectors across student layers; a decoder combines the collected evidence with the projected CLS into one Restored Token. A training-only Local Evidence-Aware Director (LEAD) further guides forensic readout. Rather than matching teacher layers directly, RELIC fixes a residual coordinate system first and then trains collection and readout in stages; a controlled ablation shows that this separation avoids unstable joint transfer. At inference, the teacher and LEAD are removed, leaving a representation usable for standalone classification or plug-and-play detector replacement. On AIGIBench Setting II, RELIC reaches 85.3% domain-macro accuracy, compared with 77.6% for AIDE, 82.3% for DGS-Net, and 75.5% for RA-Det with its original DINOv3-L interface; it reaches 91.4% on Chameleon. Probes and corruption tests further show that the recovered residual carries usable complementary evidence.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.