- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The present theoretical claim and empirical results about how their SSG technique removes the subliminal learning effect. Description of SSG:
- Generate a sample from the teacher
- Generate N rephrasings
- Get the logprobs of these N rephrasings using the teacher
- If the N rephrasings are similar enough in their logprobs, keep the sample from the teacher; otherwise, loop back to the start
- Then generate a new final rephrasing of the sample and train on it
Critical strengths
Strong positive points
- Target a real problem: subliminal learning
- Theoretical results may be sound or useful (this remains uncertain)
Weak positive points
- The decomposition of the subliminal learning problem into within- and across-orbit effects
- SSG seems like a simple technique
Critical weaknesses
Strong negative points
- The CI in their empirical results is misleading. It appears to be computed with 50x50 data points, assumed independent, while actually there are only 50 independent prompts (and 50 orderings). Their real CI could be several times larger than reported, and they don't precisely describe how they compute their CIs.
- The "validation" through empirical results is misleading. The empirical results show very weak effects, with no meaningful baselines and a single setup (1 teacher, 1 student, 1 eval, 1 random seed). The abstract and paper should not claim that SSG is "validated" through empirical results; instead, they should report a single preliminary experiment with null results. E.g., they observe a reduction in subliminal learning from 82% to 77%, but their CI ranges are only separated by 1%, while they underestimate their CIs by several times.
Questions for authors
Additional things to clarify or fix:
- The paper seems to introduce SSG as a solution while the empirical results are close to the null effect (a single preliminary experiment with a small effect), and it is impossible to evaluate relative to other techniques since they are not included.
- The "AI DISCLOSURE" section does not seem to contain all the required items listed in https://iclr.cc/Conferences/2027/AIPolicyForAuthors
- Clarify the cost of the SSG in the contribution or abstract. If it multiplies the inference cost by x16 or more (e.g., when using 15 transformations), then this is important to know.
- Describe SSG in simpler terms early in the paper