- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies input reconstruction from native Nemotron-3-Embed-8B passage embeddings obtained through mean pooling. It shows that most parameter configurations for several Transformer families map unique inputs to unique mean-pooled embeddings (generic spherical injectivity), without covering trained checkpoints or an efficient reconstruction algorithm. The method presented to reconstruct input from mean-pooled embeddings combines a learned base inverter, target-conditioned prefix guidance, residual-conditioned iterative correction, and victim re-embedding. Empirical evidence shows that 8-token inputs can be reconstructed; analysis of deteriorated reconstruction capabilities at longer input lengths shows that the correct text may never be generated. Analysis of component contributions and numerical ambiguity is also performed.
Critical strengths
- The proofs presented are insightful
- The questions are structured well: is the text unique in theory, is it separable numerically, and whether search can find it
- Modern 1-8B parameter embedders and base LLMs are tested
Critical weaknesses
- Vec2Text achieves 92% at 32 tokens, but the method presented in this paper gets 0% at 32 tokens for raw pooling on out-of-domain data. This difference makes sense given that the domain, training scale, inverter design, and victim models differ, but it is not clear whether this is because these victims are harder to invert or whether the method itself is weaker.
- The Vec2Text-style re-implementation is compared only on Llama-2, which is the easiest victim (99.7% EM in Table 1), so it is not clear how the author's method compares to Vec2Text on other, more difficult victims.
- No hyperparameter search is reported. The paper attributes the length cliff to budget but does not vary the budget at the longer lengths where recovery fails; so this may be a limitation of the configuration and not the method, and the claim may be unsupported.
- Sampler seeds are varied, but the sampling method itself and its parameters are not. It is unclear what effect the sampling method has on the accuracy of the attack. In addition, the failure analysis says that the right answer is never generated, which may be affected by sampler parameters.
- The attacker's training seeds are fixed, so it's unclear how much effect training noise has on the accuracy of the attack approach and other results.
Questions for authors
- Can the authors re-train the original Vec2Text and evaluate on native Nemotron-3 at 8 and 32 tokens?
- On native Nemotron-3, how does performance change with search budget (rounds, beam) and sampler settings (temp, top-k, etc)?
- Can Table 3 be repeated with Nemotron-3 or a setting where there is more room for a difference?
- How would the results vary when the attacker is trained with multiple training seeds?
Critical Weaknesses 1:
Critical Weaknesses 2:
Critical Weaknesses 3:
Critical Weaknesses 4:
Critical Weaknesses 5:
Questions For Authors 1:
Questions For Authors 2:
Questions For Authors 3:
Questions For Authors 4: