- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
HCD organizes behavioral, program, circuit, and state supervision and selects an adequate teacher-student correspondence by description length. Controlled tasks and generated arithmetic show promising intervention agreement, but the central selection claim and the added value of causal supervision need stronger empirical support.
Critical strengths
S1. The research question is well motivated: ordinary output agreement need not preserve responses to changes in intermediate computations.
S2. The framework separates training objectives from common evaluation requirements and explicitly restricts the correspondence family, improving conceptual clarity.
Critical weaknesses
W1. The key selection contribution is not empirically demonstrated in sufficient detail. Candidate code lengths, selection-error bounds, and adequacy decisions are missing, so it is unclear whether UCMDL meaningfully selects among several adequate alternatives.
W2. The incremental benefit over existing causal distillation is unresolved. Comparisons with behavioral KD alone do not establish what hierarchical selection adds beyond interchange-intervention or circuit distillation.
W3. The pretrained experiments are difficult to reproduce and assess statistically. Exact intervention sites and selected circuits, dataset sizes, optimization details, per-task results, and run-to-run uncertainty are not specified adequately in Appendix D.
W4. The evidence supports only a narrow form of sufficiency. Full selection is tested on two known-program tasks, and state-response evaluation includes fitting alignments to teacher state targets. Broader transfer claims require separating learned student behavior from post-training correspondence fitting.
Questions for authors
See Weaknesses