- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a neuroevolutionary approach for learning both effective and interpretable models for multi-class image classification problems. It first trains a neural newtork model as a teacher. Then, it takes the encoder part, and for each encoded feature it uses GP to learn a symbolic model representation - optimising faithfulness and interpretability. Then it uses LLM judges to judge the symbolic model interpretability for different groups of users.
Critical strengths
Explainable AI especially in medical domain is important.
The proposed method is simple and sound.
The paper is well written and the method is well explained.
Experiments and analysis are relatively comprehensive.
Critical weaknesses
I don't feel that the claimed interpretability is fully verified, as only LLM judges are used for evaluation. Real user testing and survey would serve the purpose better.
The GP fitness performance (e.g., convergence curve, model size/complexity) should be shown as well.
The explanation is for the encoded features, but not clear how these features affect the final prediction. One should also explain the behaviour of the prediction head (RF, SVM, LR) from the encoded features to the outputs.
The technical challenges and novelty are marginal.
Questions for authors
For GP parameters, how are crossover, mutation rates set? The tournament size of 7 and Elite size of 16 look very large compared with the population size of 144. Is your GP very greedy? How is the fitness convergence curve look like?
GP is trying to maximise the faithfulness of the explanation. Have you compared the feature values from the neural network and the GP outputs and how faithful they are?
How do you explain the prediction head (although they are lightweighted)? Do you only explain the relationship between inputs and the latent features?
In addition to LLM judges, have you asked real users and see how well they think the explanations are?
There are many post-hoc explanation methods for neural network, especially for image classification tasks. How to show that your model is more explainable than them?