- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes LAPCE, a class-level counterfactual explanation method for molecular GNNs. A chemical LLM, fine-tuned with SFT and PPO, proposes molecular fragments for deletion. Each fragment is then tested on all molecules of the same class, and a fragment counts as valid for a molecule only if deleting it flips the GNN’s prediction. Finally, a selection step builds one ordered list of fragments that covers as many molecules as possible at low cost. On four molecular datasets, LAPCE achieves higher coverage and lower cost than five baselines.
Critical strengths
1.Novelty. Unlike prior LLM-based work that generates whole counterfactual molecules for single instances, LAPCE uses an LLM to propose reusable deletion fragments for class-level explanation.
2.Verified and reusable explanations. Each fragment is accepted only if deleting it actually flips the GNN’s prediction, and the selected fragments form one ordered list that serves any explanation budget without reselection.
3.Thorough experimental reporting. The paper includes bootstrap confidence intervals, results on five GNN backbones, step-by-step ablations of the proposer and selector, and a detailed breakdown of computational cost.
Critical weaknesses
1.Necessity of the LLM is unclear. The LLM requires 8 to 13 hours of SFT and PPO training per task and is partly fine-tuned on BRICS fragments, yet BRICS is the only non-LLM strategy compared for candidate proposal, so cheaper alternatives such as frequent or randomly sampled substructures should also be tested.
2.Only one LLM family is tested. All experiments use ChemLLM, so it is unclear whether the method works with other LLMs, and without comparing a general-purpose LLM, the claimed benefit of chemical knowledge is not verified.
3.Modest gains at high cost. The gains over RLHEX are modest, while LAPCE spends several hours per task on verification in addition to LLM training, and since no baseline runtime is reported, it is unclear whether such small gains justify the extra cost.
4.Evaluation relies on a single distance measure. All methods are evaluated only with WNode, which LAPCE directly optimizes, while baselines such as RLHEX are trained with other distances, so results under other measures such as graph edit distance should also be reported.
Questions for authors
(1) Can you compare with LLM-GCE, the most closely related LLM-based method, using the same pooling and selection procedure as RLHEX?