- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates whether counterfactual-response agreement in policy distillation can be improved through interpretability-guided feature selection and data augmentation. It introduces an operational feature-validation gate and intervention consistency (IC), a metric measuring teacher–student agreement in action changes across matched state pairs. Experiments across PPO and SAC teachers show that counterfactual augmentation improves IC, but ordinary off-policy sampling controls recover most of the gain. A subsequent endpoint-matched analysis finds strong agreement between IC and conventional action error, without establishing additional diagnostic utility. The paper also shows that restricting surrogate inputs to validated concepts can exclude control-relevant information, while a predictive sufficiency check provides only a task-specific repair. Overall, the contribution is primarily an empirical analysis of the limitations of interpretability-guided policy distillation.
Critical strengths
-
Careful experimental controls. The study includes matched-budget comparisons, multiple sampling controls, fresh counterfactual pairs, and teacher-level statistical analyses that help distinguish ordinary distribution coverage from handle-specific effects.
-
Transparent reporting of negative findings. The authors explicitly acknowledge the limited practical benefits, metric limitations, implementation corrections, and post-hoc analyses, providing a useful empirical assessment rather than overstating the results.
Critical weaknesses
-
Limited independent utility of IC. Large IC differences mainly arise between policies with substantially different control competence. The endpoint-matched analysis shows strong correlation with ordinary action error and no established additional diagnostic value.
-
Limited interpretability-specific gains. CF augmentation improves fresh IC by 0.097, but random pairs already achieve 0.077, without a demonstrated advantage in closed-loop return or robustness. The novelty also needs clearer positioning relative to prior counterfactual and concept-based policy distillation [1,2].
-
Insufficiently validated feature selection. The response-based gate can exclude control-critical variables, substantially degrading Pendulum performance. The proposed sufficiency check repairs this case but does not establish a general solution, while matched state pairs do not isolate individual causal effects.
-
Restricted generality and incomplete cost accounting. The conclusions rely on small teachers and mostly CART surrogates, including Hopper trees that fail closed-loop, without testing augmentation on stronger surrogate classes. Equal fitting-label budgets also exclude substantial additional teacher access, and several important controls are post hoc.
[1] Counterfactual Explainer Framework for Deep Reinforcement Learning Models Using Policy Distillation
[2] Interpretable Deep Reinforcement Learning Via Concept-Based Policy Distillation
Questions for authors
Given the substantially stronger performance of the mixture-of-linear-experts baseline on some tasks, is there evidence that the sampling attribution results extend beyond CART trees to more competent surrogate policies?
We're very grateful to the reviewers for their time and for the careful reading they've given the paper. The paper doesn't clearly state its question or key takeaways, and several terms are used before being defined. We agree with that and have decided to withdraw the paper and rewrite it.
Regarding the ethics flag, we used an LLM assistant and disclosed this in the AI use statement (p. 10), as required by ICLR policy. All reported numbers were checked against the experimental outputs, and all references were checked against the sources. We accept that the writing fell short of the standard reviewers should expect, and the revised version will be rewritten by the authors.