- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether erased embeddings are actually safe. It shows that evaluation by existing methods depends on the probe and how it is fitted. The authors run a sweep over multiple encoders, datasets, and three erasure methods with two attack families: family R (logistics, MLP) and family T (gradient-boosted trees, k-NN). Their results show significant decision disagreement across attack families. The main point is if the owner release the same rows of embeddings that are used to fit the eraser, the embeddings may quietly handle the private attribute to a different probe (e.g, a tree). The paper recommends data splits (fitting the eraser on disjoint rows) so that it is easier to interpret erasure results.
Critical strengths
A shallow tree catching what the linear probe misses is interesting (Tied coordinates finding about linear erasure handling the attribute to a shallow tree). The linear probe accuracy can be easily read as safe due to below-chance accuracy, but it is due to the same fitted rows. It reinterprets below-chance accuracy. The fix is cheap and actionable.
The experimental controls are careful such as equal-size attacker, floor correction measures, a designed table, replication on new cells, a matched-n control, etc. And it tested a published conclusion directly.
Critical weaknesses
Contribution 1 is mixed with the protocol and the phenomenon. A similar work by (Snoek et al. 2019, https://www.sciencedirect.com/science/article/pii/S1053811918319463) in neuroimaging data shows that confound regression produces a negative bias below-chance in realistic cases, the bias disappears when the confound regression is done inside cross-validation fold. It is structurally similar to fitting on disjoint rows. The novelty will strengthen if the paper explains what is new in concept erasure settings. There is also a similar recent work in vision-language models about nonlinear bias survival (https://openreview.net/pdf?id=x45Ho75YmJ). The core measurement overlaps, but the settings are different.
The presentation is difficult to read. There are a lot of results. The highlights are too short. They are not easily understandable. The main content is not standalone. For example, in section 4, the ten statements (H1-H10) are from the appendix, but they are referenced throughout the section. Appendix should be additional material not the essential parts. In appendix, there are a lot of code scripts references, but there is no code attached in the supplementary material or anonymous repo.
Questions for authors
Does disjoint fitting remove the disagreements in all cells, or only most?
Thank you for the Snoek et al. reference, which we had missed; it is now cited and the novelty statement is rewritten around it.
Snoek et al. (2019) and what is new here. The structural match is real: regressing a nuisance variable out of all rows before cross-validation biases accuracy downward, below chance in the limit, and fitting inside each training fold removes the bias. Our identity (ii) is the case of that analysis in which the variable regressed out is the attribute itself, and disjoint fitting is the per-fold fix; §2 now says this in so many words, with Jamalabadi et al. (2016) for below-chance small-sample classifiers. What those accounts do not contain: the exact form of the fit at n≤d+1 (in-sample LEACE subtracts each row's own class centroid); the opposite-sign effect, by which the same fit makes the attribute more readable to a depth-4 tree through tied coordinates (0.786 above majority on ResNet-50 FairFace against 0.104 under disjoint fitting); the consequences for release decisions and method rankings; and the point that in the release setting the in-sample fit is not an analyst's slip but the deployment, so the per-fold fix changes what is released. Singh & Hegde (2026) is now cited as concurrent work: it measures the stronger-probe fact on vision-language embeddings and does not involve the fitting protocol.
Code. You are right that nothing was attached: the archive (code, dated analysis specifications, recorded results, and the scripts that produce every number, table and figure) was only linked, although the statement called it supplementary material. It is now attached to the revision as supplementary material, including the scripts and results of the new disjoint-fit arm.
All cells or most? In the controlled sweep, all: with each family reduced to its strongest member, 10 of 28 settings disagree under in-sample fitting and 0 of 28 under disjoint fitting (H9). On the designed table we have now run the arm you ask about, pre-registered before the run (revised PDF, §4 "Disjoint fitting on the flipped cells" and Appendix X10): the 19 cells that carry the 22 flips, same seeds and released rows, eraser fitted either on the released rows or on other rows of the same cache. In-sample, 19 of the 22 flips reproduce. Under disjoint fitting 2 of those 19 disagreements remain (both CelebA CLIP, and in the opposite direction: the linear family refuses what the tree family releases); the disagreement rate falls by 0.49 [0.25, 0.74]. So on the designed table the answer is "most, and the remainder is not the in-sample signature". Two honest limits. First, on the 8 ResNet-50 pairs the families agree under disjoint fitting but both refuse (R 0.06–0.16, T 0.07–0.18 above floor); we had pre-registered that T would fall below the threshold and it did not, so the erased ResNet-50 embeddings leak under both families rather than being safe. Second, the arm rests mainly on LEACE: 10 SPLINCE and 14 LEOPARD cell–arms do not reach LEACE's task cost under disjoint fitting on enough seeds and are excluded by the pre-registered rule.
H1–H10. Now tabulated in the main text (Table 1) and named where used; the gate study's detail is in an appendix.