acceptodds
Withdrawn from ICLR 2027

A Neuroevolutionary Symbolic Approach to the Interpretability–Performance Dilemma

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Neural networks achieve unmatched performance in image classification, but their reasoning remains opaque. Genetic Programming (GP) offers a more transparent alternative by evolving explicit programs, yet it struggles to scale to high-dimensional multiclass image tasks. We address this gap by distilling the latent space of a trained neural network into independent GP symbolic programs, then recombining their outputs through a lightweight classifier head. By training GP students on image-to-latent subobjectives, the proposed decomposition simplifies symbolic search and scales GP-based image classification. On four of the five studied multiclass medical imaging datasets, fully symbolic models are competitive with deep learning baselines. This neuroevolutionary symbolic model exposes an interpretability–performance tradeoff. More programs are typically required to approach teacher performance, but each additional program increases the executable logic to inspect. Because distilled student programs are executable and symbolic, they can be directly inspected, scored, and explained. We use LLM-based judges to score the interpretability of GP functions and function compositions for three stakeholder audiences, namely computer scientists, doctors, and patients. 50 participants of a human study, including clinicians, computer-vision practitioners, and non-experts, chose the LLM-preferred symbolic unit consistently across participant groups, judge families, and composed operation chains. Audience-specific scores induce distinct interpretability–performance frontiers, which we illustrate on BloodMNIST. Symbolic program diversity can therefore support stakeholder-aware model selection and attenuate the tradeoff by prioritizing programs tailored to a given audience. More broadly, this suggests a path toward expressing opaque neural representations as symbolic components that can be selected, inspected, and explained according to the needs of different stakeholders.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Genetic Programming, Interpretability, Neural Networks, Distillation, Biomedical

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_PJP4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a neuroevolutionary approach for learning both effective and interpretable models for multi-class image classification problems. It first trains a neural newtork model as a teacher. Then, it takes the encoder part, and for each encoded feature it uses GP to learn a symbolic model representation - optimising faithfulness and interpretability. Then it uses LLM judges to judge the symbolic model interpretability for different groups of users.

Critical strengths

Explainable AI especially in medical domain is important.

The proposed method is simple and sound.

The paper is well written and the method is well explained.

Experiments and analysis are relatively comprehensive.

Critical weaknesses

I don't feel that the claimed interpretability is fully verified, as only LLM judges are used for evaluation. Real user testing and survey would serve the purpose better.

The GP fitness performance (e.g., convergence curve, model size/complexity) should be shown as well.

The explanation is for the encoded features, but not clear how these features affect the final prediction. One should also explain the behaviour of the prediction head (RF, SVM, LR) from the encoded features to the outputs.

The technical challenges and novelty are marginal.

Questions for authors

For GP parameters, how are crossover, mutation rates set? The tournament size of 7 and Elite size of 16 look very large compared with the population size of 144. Is your GP very greedy? How is the fitness convergence curve look like?

GP is trying to maximise the faithfulness of the explanation. Have you compared the feature values from the neural network and the GP outputs and how faithful they are?

How do you explain the prediction head (although they are lightweighted)? Do you only explain the relationship between inputs and the latent features?

In addition to LLM judges, have you asked real users and see how well they think the explanations are?

There are many post-hoc explanation methods for neural network, especially for image classification tasks. How to show that your model is more explainable than them?

Reviewer_FituOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed a neuroevolutionary symbolic method that distills a neural network teacher's latent representation into independent MAGE programs and combines their outputs with a classifier head to do multiclass medical image classification. It also uses LLM judges to score interpretability for different stakeholders to build audience-specific interpretability-performance frontiers. Evaluations on a few datasets show large improvements over end-to-end MAGE and approaching the performance of neural baselines on most datasets.

Critical strengths
  • Effective decomposition of symbolic search: Reformulating the problem into independent coordinate-wise image-to-scalar distillation tasks via a low-dimensional teacher bottleneck representation is a practical and sound idea. The substantial and consistent performance gains demonstrate the value and practicality of this idea.
  • Strong empirical evaluation framework: Combining LLM scoring with human validation provides scalable scoring with human support for the relative rankings. Evaluating across fiveacross five datasets and various control settings makes the evaluation framework convincing.
Critical weaknesses
  • Missing baselines: The only baseline is end-to-end MAGE, where one program must output all class scores which is a weak baseline. A matched GP feature-learning baseline using the same number of programs, or a comparison with programs distilled from teacher logits instead of latent coordinates is needed. Ablations on the teacher (such as varying the bottleneck dimension or backbone sensitivity) are missing.
  • Insufficient comparison and justification against interpretable alternatives: Comparisons with established interpretable image classifiers, such as prototype-based networks (e.g., protovit, protopformer, …) or Concept Bottleneck Models would help contextualize the achieved accuracy and interpretability tradeoff. It remains unclear to me whether the proposed approach offers a favorable tradeoff relative to existing interpretable alternatives.
  • Insufficient validation of the complete model's interpretability: The human study only evaluates individual primitives and fixed 4-operation chains, whereas the proposed selection framework relies on simple mean or minimum aggregations to score complete programs and an additive cost to characterize program subsets. These aggregations do not account for program length, graph structure, or the cognitive load of understanding how multiple programs interact through the downstream classifier head. Consequently, there is no human or task-grounded validation showing that the final assembled models are actually interpretable to the target stakeholders.
Questions for authors
  • As the interpretability–performance curves for 3 groups overlap heavily and trace similar trajectories, could this partly reflect similarities in the LLM scoring to different items rather than meaningful audience-specific preferences? Also, as different subsets reach similar BACC, are the programs largely redundant?

  • What is the input resolution the MAGE programs use, and is it the same input the teacher sees? This affects how to interpret the accuracy gap and the compute cost. Did you also try other image resolutions to see the effect? How sensitive are the results to the bottleneck dimension of 16, particularly for more complex datasets?

Suggestion: Adding a qualitative case study that contrasts a program selected for a “Doctor” versus a “Computer Scientist” on the same image would strengthen the intuition for the reader.

Reviewer_4nEfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an approach for symbolic distillation of neural networks trained for multiclass medical image classification. A ResNet-18 teacher is fine-tuned to 16-feature bottleneck and each latent feature is independently distilled into symbolic programs. The paper also proposes an interpretability framework which considers multiple stakeholders and uses LLM judges to score.

Critical strengths

The spearman rank fitness for symbolic distillation is unique and novel to the best of my understanding. Though the student model no now longer produces the same values, it maintains the monotonic order of outputs. This is helpful to future researchers in this area and one immediate use of this approach is that a student model with high fitness can still reproduce the output if it is paired with a monotonic function (either an explicit expression or an arbitrary monotonic curve works).

The paper uses a multiple stakeholders angle for analysis and scoring, followed by a human study with professionals. This greatly enhances their interpretability claims if the human study asks important interpretability questions.

Critical weaknesses

Empirical evaluation of prediction performance is weak. Lack alternative symbolic feature-learning baselines that are cited in the paper. Currently, the paper is lacking these results.

I also cannot find any error bars or significance tests for the main numerical results analyzed.

Prompt template derivation not described. Currently, it seems ad-hoc and the prompt is not clear, especially in the scoring, where only levels 1, 5, 10 are defined without a clear discussion of how intermediate scores should be interpreted.

The paper seems to have blurred the lines between model transparency and true interpretability. The 16 latent features used as a proxy makes it difficult to truly interpret the results since the 16 features still have to do undergo additional processing. The paper also clarifies that "Medical professionals have not reviewed the generated explanations.". Thus, it is not convincing that what the work has measured is truly interpretable to the professionals.

The title appears misleading and should specify the work is for images only. The paper itself has also recognized that symbolic distillation of neural networks is not new, and there are also many other works on symbolic distillation of neural networks that the paper has not cited, in venues like ICML, GECCO. The novelty of the paper lies in applying this for images, and I think it should be reflected in the title rather than seemingly making a claim which multiple prior works have already tackled.

Questions for authors

Please address the weaknesses above.

Reviewer_foiQOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper trains GP programs to approximate the 16 latent features of a ResNet teacher, then combines their outputs into a classifier. This works much better than the direct multiclass GP baseline on five MedMNIST datasets. The authors also ask LLMs to rate how easy the symbolic operations would be to understand for computer scientists, doctors, and patients. They then run a human study to check whether people agree with these rankings.

Critical strengths
  1. I like the idea of using a neural teacher to help build an interpretable classifier through symbolic programs.
  2. The classification results are promising across the five MedMNIST datasets, with large gains over the direct multiclass GP baseline. The final model no longer needs the neural encoder, and its intermediate computations can be inspected.
  3. The authors include several useful checks, such as using fewer searches and comparing random and structural program orderings. The human study accounts for repeated responses from each participant when computing confidence intervals. In 76.8% of function comparisons with a clear preference, participants agreed with the LLM ranking. This gives some support to using these scores to judge how easy individual operations are to understand. The authors also acknowledge the weaker OCTMNIST results and the limits of their evaluation.
Critical weaknesses
  1. I am not yet convinced that the comparisons show what latent decomposition itself contributes. Appendix M shows that a dictionary built from 16 distillation searches beats even the best of 16 end-to-end runs. That is useful, but the model structure and optimization still differ. I would like to see a GP feature dictionary trained without latent targets and combined with the same classifier. This would help separate the benefit of using ResNet's learned representations from the benefit of the proposed decomposition.

  2. Understanding a few operations does not necessarily mean understanding the full model. Participants compare individual operations and four-operation excerpts, rather than complete programs or classifiers. Figure 2 mainly shows how BACC changes as more programs are added; it does not measure how well people understand the model or report the defined audience cost. I therefore do not think it yet shows that the selected models are easier to understand at similar accuracy.

  3. It is unclear whether tailoring the scores to each audience actually helps. Agreement above chance is encouraging, but would a general ranking, or even a ranking intended for another audience, do just as well? Different rankings select different programs, but that alone does not show that those programs better suit the intended users. The existing human responses could be used to compare matched, pooled, and mismatched audience scores directly.

  4. The results seem quite dependent on the dataset. The difference between BloodMNIST and OCTMNIST makes me wonder which properties of the data or domain determine when this approach works well.

Questions for authors
  1. Does latent distillation still help when compared with a supervised GP feature dictionary using the same classifier head, number of programs, primitives, and search budget? What happens if the targets are the teacher's logits instead?
  2. Could you compare a few complete models with similar BACC and test whether audience-based selection helps users trace predictions more accurately or quickly than structural or feature-importance selection?
  3. On the existing human responses, do matched-audience scores predict preferences better than pooled or mismatched scores? Please account for repeated responses from each participant when reporting uncertainty.
  4. Where would this type of classifier be useful in practice, and how well does the approach carry over to different application settings?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.