acceptodds
Withdrawn from ICLR 2027

EVIL: Evolving Interpretable Algorithms for Zero-Shot Inference on Event Sequences and Time Series with LLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We introduce EVIL (EVolving Interpretable algorithms with LLMs), an approach that uses LLM-guided evolutionary search to discover simple, interpretable algorithms for dynamical systems inference. Rather than training neural networks on large datasets, EVIL evolves pure Python/NumPy programs that perform zero-shot, in-context inference across datasets. We apply EVIL to three distinct tasks: next-event prediction in temporal point processes, rate matrix estimation for Markov jump processes, and time series imputation. In each case, a single evolved algorithm generalizes across all evaluation datasets without per-dataset training (analogous to an amortized inference model). To the best of our knowledge, this is the first work to show that LLM-guided program evolution can discover a single compact inference function for these dynamical-systems problems. Across the three domains, the discovered algorithms are often competitive with, and even outperform, state-of-the-art deep learning models while being orders of magnitudes faster, and remaining fully interpretable.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM-guided program evolution, evolutionary program synthesis, algorithm discovery, interpretable machine learning, zero-shot inference, in-context learning, amortized inference, dynamical systems, time series, temporal point processes

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_go4pOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes an original idea: can we use an evolution framework to have LLMs write Python functions that converge to a best estimate for various timeseries-related tasks. They use OpenEvolve, various LLMs, handwritten prompts, and 3 different tasks: temporal point processes, Markov Jump Processes, time series imputation.

The approach sounds interesting (at least to me), the experimental design is sound, the paper is clearly written with good figures and a lot of data, and the findings are convincing. In my assessment, it is a solid paper, but I am not sure it is significant and substantial enough to warrant publication at ICLR. More comments below.

Critical strengths
  • interesting approach (wrapping LLMs in an evolution search loop), which seems extendable to a broader class of problems
  • compelling results, especially pertaining to the MJP
  • good presentation overall, and the authors clearly care about making their contribution useful to the community; for example, I appreciate Appendix I: Applying EVIL to a New Problem.
Critical weaknesses

The main weakness I see is that I am not sure the setting of time series prediction/interpolation makes a very strong case for the more general, and in my view more interesting, framework of LLM-guided evolutionary search. In particular, the authors present a lot of results, and very detailed experiments (which is good), but I regret that they don't discuss in more depth the broader implications of their methodology and findings. The conclusion is just a few lines. In its current form, this paper seems more suited for a workshop (more narrow in scope) than for the general ICLR conference.

Questions for authors

See weaknesses. Also, could you clarify your setup, as described in Appendix K? I don't understand how the "weighted ensemble" of LLMs is created and assessed.

Reviewer_osBxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed EVIL, a LLM-guided evolutionary search approach to discover interpretable programs for zero-shot inference across datasets. The approach was evaluated on three different settings: 1. next-event prediction in temporal point processes 2. rate matrix estimation for Markov jump processes, and 3. time series imputation. The evaluation protocol is clean with clearly held-out test samples. The discovered algorithms perform faster inference than their neural baselines and stay interpretable.

Critical strengths

The idea of using LLM to guide evolutionary search for temporal modelling problems is novel and inspiring. The evaluation is thoroughly performed across settings and the paper is pretty well written.

Critical weaknesses
  1. The empirical results are generally mixed, in many cases the proposed approach did not outperform the simple baselines, and it is not clear when and why.
  2. I believe the inference speedup is one of the core contributions of this paper, given the mixed results. However, the comparison in the abstract "Inference is roughly two orders of magnitude faster than the neural baselines" overly focused on the neural baselines, while neglecting the classical methods. As shown in Table 19, EVIL is consistently slower than classical methods, while sometimes the classical methods do outperform EVIL (e.g. ACD is best on STACKOVERFLOW and RETWEET, as shown in Table 1 while being faster) . I think this perspective needs to be explained better to make the contribution more convincing.
  3. The actual zero-shot transferability is unclear: it is hard to assess it without knowing the actual distribution shift between the data used for evolve and the held-out test sample. see questions as well.

Overall, the evidence for the need of evolutionary algorithm discovery across all three tasks is not sufficiently strong to me.

Questions for authors
  1. In figure 1, could you provide an example of the trivial baseline code? In general it would be very helpful to have a detailed explanation of each component in the caption.
  2. Can you please clarify the tradeoff between speedup and performance, especially also in contrast to the classical methods?
  3. How different are the evolution and test distributions? Even exact test samples are unseen, repeatedly scoring and optimising for subsamples from the same distribution would create a benchmark-adapted program instead of a general purposed algorithm. I think clarifying this perspective is very important to actually evaluate the transfer capability of the proposed method.
Reviewer_864cOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a technique for completing time series. The method searches over Python programs that complete a time series, and argues that these results are interpretable, and perform similarly to neural approaches.

Critical strengths

The idea of searching for a program to explain and predict a time series is neat, although this is absolutely not new.

Critical weaknesses

The idea proposed in this paper is neat, but the paper remains superficial, in both the discussion of the method itself, and the integration to the broader academic literature. The writing of the main text is too sparse in information, with crucial details left to the appendix, which is too long. The reader is left to piece together what the actual argument and benefits are. I would argue that it does not meet the standard for an academic contribution that should be expected at NeurIPS.

The description of the method is very superficial. The method that is proposed is a combination of various different existing packages put together. This is not a problem in itself, but a proper scientific paper would give a more complete insight into how the methods that are cited work.

The integration and positioning of this work in the broader scientific literature is also poor. The idea of program search is very old, and highly studied in AI, e.g. inductive logic programming. Very little is cited. Deep concepts are alluded to, but cited very superficially, e.g. line 730 where short programs are argued to generalise. There is a whole literature on Algorithmic Information Theory that is not cited.

Crucially, for such a simple method, the empirical evaluation must be the cornerstone of the argument that this method is useful. While the numbers look good for EVIL, the datasets themselves are treated as black boxes. While this is often the case in machine learning, for methods that are increasingly simple like these, it becomes more crucial that it is argued that the baselines are appropriate for solving these datasets, and that the method that is proposed delivers meaningful results.

Questions for authors

How would the datasets that you propose be modelled if one were actually interested in using the predictions? Can you evidence this with other papers? Does this method give results of a quality that would actually be useful for these datasets?

What is the claim about how this method could be used now, or how it could be used in the future if it were developed further? What performance would it need to achieve to be useful?

For simple methods like these, which consist of putting together existing tools, the acceptance must depend strongly on the result actually solving a problem. It is not surprising that LLMs can output code that does something. The key point is that it must do something good for it to be interesting. Making a point on synthetic datasets is completely fine.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.