- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies the use of SAEs on OOD detection problem. Authors start with a negative result on SAE OOD units, linking it to SAEs nonidentifiability across different seeds. Then they make the observation that SAEs' usefulness for OOD detection comes mainly from reconstruction gap, rather than specific OOD units. SAE encode-decode is a lossy transformation that mainly results in a confidence drop on the linear probe estimator. Overall, the paper presents a systematic study to investigate these effects.
Critical strengths
- Given the existing works' claims, demonstrating a negative result is valuable. Stressing unidentifability of SAEs challenges the existence of specific OOD units for detection problem.
- Understanding the mechanism of why SAE work for OOD by checking its reconstruction effects is insightful.
Critical weaknesses
- Paper uses a lot of terminology, is very dense, repetitive and hard to follow. Authors explicitly acknowledge substantial AI assistance for drafting but I personally do not think this is at a publishable level.
- Evaluation protocol is far form the modern OOD detection evaluations. There is no structured near and far OOD analysis and the used benchmark seems very outdated and at the saturation level.
- Non-identifiability of SAEs and instability of its features are already well-known. Cited works with SAE OOD units are very recent preprints which do not show strong performance across architectures and datasets. Together, this partially challenges the motivation.
- Conclusions are underwhelming. Methodology is not well tested across benchmarks. Even then, this acts more as an observational result rather than a solution to an existing problem. Which itself could be valuable when the results are counter intuitive or well tested. I think the work provides a useful preliminary study but its conceptual insights seem limited for a conference paper.
Questions for authors
- I am curious about the performance of SAE as a detector on more recent standardized benchmarks like OpenOOD. Could the authors run additional experiments across different architectures to check if reconstruction effects are consistent?
2 comments
Part 2/2: Protocol, motivation, conclusions and writing
Evaluation protocol (W2). You were right that our benchmark was saturated, and the OpenOOD run showed one reason. CIFAR-10 images are 32 px, but several of our OOD sets were used at full resolution, which gives feature-space detectors a resolution cue. On OpenOOD's Textures and Places365 lists at stored resolution, SAE reconstruction reaches 1.000 and 0.995 AUROC (PCA and KNN behave the same way; MSP does not); with OpenOOD's 32-px step these fall to 0.961 and 0.834. The revision replaces Table 1 with the OpenOOD near/far table. The round-trip result survives the same correction: with ImageNet-O brought to 32 px, the drop is 62% versus 11% on ID (66% with CLS input), against 2% for matched noise; with CLS input and SAEs trained on all 50,000 images it is 54% versus 5%.
Motivation (W3). We agree that SAE non-identifiability is known (Paulo and Belrose, 2026; Fel et al., 2025) and that the SAE-OOD works we cite are recent (two of them preprints); none of them claims seed-stable OOD units (what each reports: our reply to 8Wfp, part 4). The motivation does not rest on them: splicing SAE reconstructions into a model and measuring the change in its loss is a standard SAE evaluation (Gao et al., 2025), and the round-trip drop is that evaluation read through a classifier's confidence. The revision cuts the unit-level part to one stability table and the result that five fixed units detect ImageNet-O within one SAE, moves the per-unit analyses to an appendix (Section 6 keeps a short summary), and presents it as confirming known instability in this pipeline. The finding we consider new is about what the confidence signal measures. In our experiments the round-trip drop concentrates on OOD inputs when the reconstruction is fit on a narrow ID and read by a head whose label space is much wider, and its size varies with the backbone and with the SAE: large for ViT-B/16 and DeiT-B/16 on CIFAR-10 (39-83% on the OpenOOD means with the paper's SAEs (trained on a 20,000-image subset), 20-43% with SAEs trained on all 50k images), smaller for ResNet-50 (10-17%) and with ImageNet-100 as ID (about 10% on ImageNet-O, under trained heads), and small at ImageNet-1K (6-7%). PCA-256 fit on the same ID training set gives a similar drop with the paper's ViT-B/16 SAE (73% against 70% on ImageNet-O) but not everywhere: 5% against the SAE's 14% on ResNet-50, 54% against 6% at ImageNet-1K, and 73% against 53% with the all-50k SAEs. Reading such drops as evidence about SAE features therefore needs these controls, and the audit provides them.
Conclusions (W4). We agree that the result is observational and diagnostic rather than a solution, and the new conclusion says so plainly, with three takeaways: SAE scores trail standard post-hoc detectors on OpenOOD; the units that detect OOD inputs are mostly specific to one training seed; and a round-trip confidence drop should be read only beside the noise, PCA and matched-head controls, because its size depends on the head and on the data the SAE is fit on. It is now tested on the OpenOOD CIFAR-10 and ImageNet-1K benchmarks, in addition to the original sets, and on three backbones (two ViTs and a CNN).
Writing (W1). We accept this. The revision rewrites the abstract, introduction and setup, and cuts repetition in the results sections and the conclusion. The task and the confidence measure are defined before any result, terms such as "seed-idiosyncratic" are removed or defined, the long name of the effect is dropped, and the audit is given as an algorithm box, with the equation of each score in Section 3.
References: Paulo and Belrose (2026), ICLR; Fel et al. (2025), ICML; Gao et al. (2025), ICLR.
Part 1/2: OpenOOD v1.5 results and the round-trip drop across backbones
Thank you for the review. Your question asks for exactly the evaluation the submission lacked, so we ran it.
In short: SAE reconstruction is not a strong detector on OpenOOD v1.5, and the round-trip confidence drop has the same direction on the three supervised backbones we ran, with a size that depends on the backbone and the SAE.
Detection. OpenOOD v1.5 CIFAR-10 benchmark (OpenOOD's image lists, near/far groups and metric code; every image resized to 32 px on its shorter side and center-cropped to 32×32 px, then our 224-px input transform); ViT-B/16, mean pooling, the paper's CIFAR-10 SAE. MSP, Energy and ViM use a CIFAR-10 linear probe on the frozen features; MSP and Energy use it with temperature scaling fit on in-distribution (ID) validation data, and in brackets without it (T = 1, as in OpenOOD). AUROC / FPR@95. We keep frozen ImageNet backbones instead of OpenOOD's training protocol, so these numbers are not comparable with its leaderboard (our reply to 8Wfp lists every difference).
Across seven configurations on three backbones (ViT-B/16 and DeiT-B/16 with mean and CLS pooling, ResNet-50, and ViT-B/16 mean and CLS with SAEs trained on all 50k images), ViM is best on near-OOD in all seven and on far-OOD in six (Energy on DeiT-B/16 CLS far-OOD). SAE reconstruction ranks 5th to 8th of 8 (4th to 8th at T = 1). It is below MSP on near-OOD in all seven configurations with the calibrated probe and in six at T = 1 (DeiT-B/16 mean: 0.895 against 0.887), and both SAE scores are below Energy in all 14 CIFAR-10 cells. On ImageNet-1K (ViT-B/16, 45,000 ID test images; mean-pooled features with the native head and a Resize(224) crop, which differ from OpenOOD's protocol), SAE reconstruction with three TopK SAE seeds reaches 0.706-0.710 AUROC on near-OOD and 0.906-0.907 on far-OOD, behind Mahalanobis (0.741), MSP (0.734) and ViM (0.721) on near-OOD and ViM (0.928) and Mahalanobis (0.923) on far-OOD.
Round-trip drop (a main-text table in the revision). Relative drop in the mean MSP of the frozen 1,000-class ImageNet head when each embedding is replaced by its SAE reconstruction, on the same OpenOOD sets (ID / mean over the near-OOD sets / mean over the far-OOD sets). Gaussian noise with the same squared error moves MSP by at most 2.1% on any of these means. Each row uses one SAE seed.
The OOD drop is 2.9 to 10.3 times the ID drop on the CIFAR-10 benchmark (3.3 and 3.8 times in the ImageNet-1K row). This ratio reaches 5, the threshold we fixed before this run, in only 4 of the 14 CIFAR-10 cells (ResNet-50 far-OOD and three cells with the all-50k SAEs), on both groups of a configuration, as the check was written, in 1 of 7 (ViT-B/16 mean, all-50k SAE), and in none of the ImageNet-1K cells; when the audit also requires a drop of at least 20 points, in 2 of 14 cells. The drop is small at ImageNet-1K (1.9% here on OpenOOD's 45,000 ID images; 1.8% on the 50,000 validation images in our reply to 8Wfp), where the SAE is fit on the head's own domain and has more capacity (see our reply to 8Wfp, part 3).
One clarification: the large drops are on the wide ImageNet head. A CIFAR-10 linear probe, matched to the ID label space, is much less affected (between -3% and 14% on the near and far means).
Reference: Zhang et al. (2024), OpenOOD v1.5, DMLR (arXiv:2306.09301).