- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper asks whether a cleaner internal representation implies a smaller causal circuit. From one GPT-2 Small checkpoint it continues training with a standard objective and with ten-step PGD on token embeddings, keeps both models competent at indirect object identification, and checks that the robust model is robust under the same attack used in training. It then compares SAE reconstruction error, how many SAE features the task attribution uses, and how many edges EAP-IG must keep to recover IOI behavior at a given faithfulness. The robust model is more SAE-decomposable and engages fewer features. Circuit size depends on the cutoff: the standard model leads or ties through 85% faithfulness, while on the primary pair the robust model reaches 90% with about 1,000 edges against 4,000, and 95% with about 2,000 against 16,000. The area under the same curve favors the standard model, 0.87 to 0.60.
Critical strengths
The comparison holds task competence fixed before it compares mechanisms, and it says in advance that representational cleanliness and circuit size are different quantities. Figure 2 gives both readings of one curve: the models meet at 85%, the standard model leads through the steep part of the curve, and the robust model pulls ahead only at high faithfulness. The paper states that no cutoff is a field standard, that the 90% and 95% counts were computed for the primary pair only, and that Greater-Than is not competence-matched because the robust model scores 0.442 there against 0.671. The representational trend, unlike the high-faithfulness gap, is repeated across a seven-point sweep and a second corpus. That is a usable result.
Critical weaknesses
The result that would support a stronger score is the edge gap at 90% and 95%. It comes from one pair, 200 prompts, and a coarse grid on which a reported count means the crossing fell in the preceding bin. Those crossings were not recomputed for the sweep or the second corpus. SAE reconstruction uses three seeds; the circuit comparison does not. Adversarial training changes more than feature entanglement, so the dissociation is shown for this training instrument on IOI, not for representational simplicity in general. The paper already states these limits. They are why the acceptance is weak.
Questions for authors
On the models already trained for the seven-point sweep, do the 90% and 95% crossings still fall in different grid bins, in the same direction, on the same 200-prompt protocol? A stable gap would move this toward a clear acceptance. A reversal would move it back to a weak rejection.