acceptodds
Under review as a conference paper at ICLR 2027

Behavioural Partitions: Evolving Piecewise Laws from Data

Abstract

Many empirical laws feature a symbolic formula that only holds within some region of the input space. Learning such laws from data is especially challenging, because different formulas apply in each region and the regions themselves are unknown. The challenge is to learn both at the same time: the regions and the formula that holds within each of them. We propose BePart, a model that learns a symbolic representation of the regions and of the law governing each region. To this end, BePart searches for a set of gates, one for each region, compared by an argmax, so that every input is routed to exactly one symbolic formula, the law of its region. In contrast to other mixture-of-experts approaches, BePart finds a symbolic representation of both: the gates and the law of each region. BePart evolves whole partitions by a genetic algorithm under Pareto selection on error and complexity. On a new suite of 38 regime problems with known partitions and laws, spanning synthetic recipes, piecewise physics and hybrid dynamical systems, BePart recovers the partition (ARI 0.91) and the complete piecewise law in 53% of the runs, where model trees and piecewise-affine regressors find isolated affine laws but never the partition, mixtures of experts recover the partition only roughly and no laws, and the best single-expression system with conditional primitives reaches an ARI of 0.63 and 17% piecewise law recovery. With a single region, the task becomes classical symbolic regression. BePart shows its strength in this case as well: under the SRBench evaluation cap it attains the best median test R2 and mean rank against the 21 published methods of the SRBench black-box suite, and on the SRBench ground-truth problems it recovers more laws symbolically than any published method at noise levels up to 1%.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
symbolic regression, piecewise regression, genetic programming, mixture of experts, scientific discovery

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_m74cOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a method for piecewise symbolic equation discovery. It employs an EM-style algorithm to identify segment boundaries (partitioning) and utilizes symbolic regression to evolve explicit mathematical expressions within each segment.

Critical strengths

The problem targeted by this work extends the boundaries of traditional symbolic regression, shifting attention toward the discovery of piecewise functions. This direction is highly valuable and relevant, as many real-world physical and engineering systems exhibit regime-dependent behaviors that cannot be captured by a single global formula.

Critical weaknesses
  1. Severe writing deficiencies and AI-generation artifacts.​ The overall writing quality is poor and fundamentally impedes readability. The text exhibits clear signs of AI-assisted generation without adequate human review: individual paragraphs appear superficially logical in isolation, but inter-paragraph transitions are incoherent and disjointed. This creates an illusion of content while leaving readers unable to grasp the paper's central thesis after reading. Basic academic writing literacy—such as constructing a clear narrative—is lacking.

  2. Unclear motivation and failure to articulate core challenges.​ The Introduction does not clearly explain the specific difficulties in piecewise function discovery, nor does it position the proposed method relative to existing limitations. A fundamental logical progression—from problem hardness to current algorithmic status to the authors' angle of innovation—is entirely missing. Readers cannot understand why this work is necessary or what exactly it contributes.

  3. Absence of a methodological framework figure.​ The paper provides no pipeline or overview diagram to illustrate the proposed approach. Given the complexity of combining genetic algorithms with EM-style gate creation, such a visual aid is essential for reader comprehension; its absence further exacerbates the readability problem.

\textit{(Note: The severity of these writing and presentation issues alone substantially undermines the paper's publishability, regardless of experimental merit.)}

Questions for authors
  1. What is the core innovation of this work? Is the application of the EM algorithm a direct, off-the-shelf adoption, or has it been specifically modified/adapted to address the unique challenges of piecewise function discovery (e.g., non-differentiable argmax routing)?

  2. What is the motivation for using an EM-style algorithm rather than heuristic methods or even a simple grid search? Specifically, since symbolic regression for different segments can be performed independently and in parallel without interference, a grid search over partition boundaries could systematically explore the space and guarantee optimality (or at least a globally optimal partition given fixed expressions). Why was this straightforward baseline not considered, analyzed, or compared against?

  3. The proportion of datasets derived from real-world systems in the experimental validation is too small. Given that the paper targets "behavioral partitions from data," the evaluation relies heavily on synthetic or engineered benchmarks. More realistic datasets are needed to demonstrate practical utility beyond controlled settings.

2 comments

Thank you very much for your feedback! First, one correction to the summary: BePart does not find the boundaries with EM and then run symbolic regression per segment. Whole partitions, i.e. gates and experts together, are evolved jointly by a genetic algorithm under Pareto selection on error and complexity. The EM-style gate step is only one of its optimizations.

Weaknesses

1. Writing and narrative

We would be grateful for examples of passages the reviewer found incoherent, so that we can fix them directly. As stated in our AI use statement, AI was used for drafting and editing, but every sentence, claim, number and citation was checked, and most of the text was written or adjusted by hand. Nonetheless, we will check our text for the issues you mentioned and improve them.

2. Logical progression of the introduction

We agree that the introduction lacks the core challenges of piecewise function recovery and a logical progression, and we have revised it accordingly. Motivation and positioning however are, in our view, covered in the first paragraph and the related work.

3. Overview figure

We agree. Fig. 1 shows the model and the shortcomings of the baselines, but not the search. The revision, uploaded during the discussion period, adds a method overview figure to Sec. 3.

Questions

1.1 Core innovation

BePart is, to our knowledge, the first model in which the partition and the laws are both symbolic and searched for jointly. The technical contribution is the search machinery that makes this joint search work, since plain variation and selection almost never create a useful regime.

1.2 Is the EM step an off-the-shelf adoption?

No. The off-the-shelf version is one of our baselines: a mixture of linear regressions fitted by EM with a logistic router. On the regime suite its ARI is 0.35, against 0.91 for BePart (srbench-cap, Tab. 1). Adapting EM to our model required solving two issues. (1) Vanilla EM takes its E-step from the router's likelihoods, but our gates are not probabilistic and the routed error is a step function. The oracle assignment (that routes a point to its best fitting expert) supplies the missing target. (2) A gate has to win the argmax against the other gates on its points. The +1/-1 target around the strongest competitor's gate turns this into a least-squares problem, so the gate is fitted in closed form.

2.1 Why EM-style rather than heuristic methods?

BePart is a genetic algorithm and therefore itself a heuristic method. The EM-style step is an additional optimization, added because random variation rarely produces a useful gate, while the EM step is cheap (with the experts frozen, a candidate gate costs one gate evaluation and an argmax).

2.2 Could a grid search with independent SR per segment guarantee optimality?

Only within a fixed family of boundaries and at high cost. Every candidate partition costs one SR run per segment, and the number of candidates grows exponentially with the input dimension and the number of segments, so the grid soon becomes infeasible even with parallelization. BePart can be parallelized as well, but even on one core a fit takes 2.6 s (default, Tab. 1). The optimum of a grid search is also guaranteed only within the grid's family and resolution, and only if SR solves every cell optimally.

2.3 Why was this baseline not compared?

Two of our baselines follow this idea. CluBS (Zdankin et al., 2026) clusters the points by the SR model that explains them and fits one PySR expression per cluster. It reaches an ARI of 0.15 at a median of 4,979 s per fit, against 0.79 and 2.6 s for BePart (default, Tab. 1). The linear model tree is a greedy version of the grid search: it tries axis-aligned thresholds and fits a model on both sides of each. Its ARI is 0.48 on the 23 axis-aligned problems and 0.16 on the 15 others (BePart srbench-cap: 0.95 and 0.86).

3.1 Why mostly synthetic or engineered problems?

Partitions and laws can only be scored where the true partition and laws are known. In measured data the regimes are generally not labelled. We therefore built 14 of the 38 problems from textbook engineering laws and hybrid systems, over their physical input ranges and with up to 10% target noise.

3.2 Evidence of practical utility on realistic data?

Real data are in the paper, but can only be scored for accuracy: the SRBench black-box suite (122 PMLB datasets, 60 of them not synthetic Friedman problems) and the 35 datasets of OpenML-CTR23 (App. D). On SRBench, BePart has the best median test R2R^2 at k=1k=1 (Tab. 2). On CTR23, allowing several cells raises the median R2R^2 from 0.776 to 0.821 (srbench-cap, Tab. 16). We agree that a case study on measured data with physically known regimes would show the practical use more directly and is part of ongoing research.

The revised paper is now uploaded, and the supplementary material and the anonymous repository are updated.

Weakness 1, writing: we went through the text again and revised the wording, mainly in Sec. 1, 3, 5.2 and 6.

Weakness 2, introduction: it now names the challenges of recovering piecewise laws first, followed by the existing methods and our approach (p. 1).

Weakness 3, overview figure: Fig. 2 shows the model and its search (p. 4).

Our reply quoted the numbers of the submitted version. In the revision the regime suite has 46 problems and BePart was run again, so the values are slightly different. The EM mixture has an ARI of 0.39 and BePart (srbench-cap) of 0.90 (Tab. 1, p. 8). The default configuration reaches 0.79 at 3.1 s per fit, and CluBS 0.16 at 5,557 s. The linear model tree has an ARI of 0.59 on the 23 problems with axis-aligned boundaries and of 0.21 on the 23 others, against 0.95 and 0.84 for BePart (Tab. 17, p. 46). 22 of the 46 problems are built from engineering laws and hybrid systems.

We would be grateful if you could take a look at the revised paper and let us know whether it addresses your concerns.

Reviewer_YE5KOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to recover both the boundaries and governing laws of piecewise functions from data. The authors propose BePart, which uses symbolic gating functions to assign each input to a region through an argmax rule. It uses symbolic expert functions to model the law within each region. A genetic algorithm searches for the gates and experts together, while NSGA-II balances prediction error and expression complexity. The method also includes gate refinement, residual based partitioning, least squares expert calibration, expression growth, and region merging. Experiments cover 38 piecewise problems with known regions and laws, along with the black box and ground truth tracks of SRBench.

Critical strengths
  1. The problem holds significant scientific and engineering value, addressing prediction, regional segmentation, and the interpretability of mathematical formulas simultaneously.
  2. The use of argmax symbolic gating enables explicit, hard partitioning, which enhances the model's interpretability.
  3. The methodology is comprehensive, featuring specific mechanisms designed to overcome challenges related to expert calibration and the creation of gating functions during the symbolic search process.
Critical weaknesses
  1. The paper lacks a discussion on the identifiability of the "true" partitions.

While the paper treats the number of regions and their boundaries as the ground truth, it fails to specify the assumptions under which these can be uniquely recovered from the input-output data. A single function may admit multiple equivalent piecewise representations. For instance, a single region could be subdivided into two, with both sub-regions sharing the same underlying formula, yielding identical predictions. Furthermore, certain piecewise patterns might be implicitly represented by a single expert function that incorporates operations such as absolute values. Therefore, a more precise objective for the paper would appear to be the recovery of a low-complexity, prediction-equivalent piecewise representation, rather than the recovery of a unique "true mechanism." The authors should further clarify the conditions for the identifiability of the number of regions, their boundaries, and the governing laws within each region, as well as how a "minimal" or "canonical" piecewise representation is defined.

  1. The setting of the number of components for some baselines may compromise the fairness of the comparison.

In Table 6, MoE, EM-based linear mixtures, and PARC use K=8K=8, whereas the ground-truth number of regions for the test problems ranges from 2 to 5; BePart, by contrast, can automatically determine the number of components. While setting K=8K=8 does not necessarily lead to trivial results, as redundant experts might never "win" on the test data, if these redundant experts end up forming non-empty regions, they could fragment the true regions, thereby negatively affecting the ARI and PER metrics. Since PER requires the recovery of the exact number of ground-truth regions, fixing the number of experts at 8 may impose a structural disadvantage on these baselines. Could the authors provide additional results to verify the fairness of the comparison?

Questions for authors
  1. The criteria for LRR and PER allow for recovered formulas to differ from the ground-truth formulas by an additive or multiplicative constant. Since this does not equate to the strict recovery of the original formulas and parameters, should the term "complete/exact law recovery" in the abstract be revised?
  2. While the gating representation is a core contribution of this paper, the independent benefit of the specific gating construction mechanism appears inconsistent. Removing the EM component leads to a significant drop in ARI, whereas using random gating splits or removing the "grow chain" mechanism actually results in slight performance improvements. Should the authors perhaps soften the claim that "all gating mechanisms are necessary"?
  3. The experiments involving engineering and hybrid systems are primarily generated using predefined formulas; for the hybrid systems, data is sampled from the state space, with known derivatives used as targets. Given the absence of real-world trajectories, potential errors in derivative estimation, sensor noise, and history-dependent switching, are these experiments sufficient to support the paper’s conclusions regarding real-world engineering applications?
  4. The paper notes that for problems governed by a single underlying rule, the default automated configuration selects only one component in just 41% of runs. Does this result indicate that the current model selection process still suffers from a significant issue of over-partitioning?
2 comments

Thank you for the careful review. Numbers and section, table and appendix references are those of the revised paper, which we will upload as soon as possible.

Weaknesses

1. Identifiability of the true partitions

We agree: the paper treated the generating partition as the truth without an assumption under which it can be recovered.

The number of regimes, the boundaries and the laws are identifiable under two requirements: every law is real-analytic on a connected open set that contains the interior of its regime (the regime without its boundary points), and the number of regimes is the smallest possible. The first excludes an expert that hides a boundary in an absolute value (a step function can be written as 1/2+∣x∣/(2x)1/2 + |x|/(2x)). The second excludes a regime cut into cells with the same law. We call the representation that meets both requirements the canonical form. It is unique up to a null set, because two different analytic functions agree only on a null set (Mityagin, 2020).

A model's canonical form cannot be constructed in general: whether two cells given by expressions are the same set is undecidable once sin⁡\sin is among the primitives (Richardson, 1968). The search algorithm therefore stays as it is and looks, as the reviewer puts it, for a low-complexity, prediction-equivalent representation. Instead, the scoring applies the canonical form on the test points, so that equivalent models get the same score whether or not they are in canonical form: cells with the same law are merged, a cell is split where an absolute value in its expert changes sign, and each law is checked on its cell only. A ground truth defined by minimal complexity would depend on the primitive set and on how nodes are counted, so we rely on the scoring to assess equivalence.

In the revision, Sec. 2 states the two requirements, and App. B gives the argument and the scoring, which is applied to every method. The bouncing ball's flight law contains ∣x1∣|x_1| and was the only ground-truth law not in canonical form, so we split it in two, which gives the problem three regimes. The absolute value is removed from BePart's primitives on the regime suite. When the submitted runs are re-scored, no fit loses its PER verdict, and at noise 0 BePart still leads every baseline on ARI, LRR and PER.

2. Fixed number of components for the baselines

We agree: with K=8K = 8 a baseline can cut a regime into several cells, which lowers its ARI and makes PER fail by the count alone. The revised scoring drops unused components and merges cells with the same law for every method. We ran the four baselines with a fixed KK in two further configurations: one with KK set to the true number of regimes, and one in which every KK up to 8 is tried and the one with the lowest validation error is chosen. Neither improves their results substantially (App. D; see our reply to Reviewer txDZ). What limits these methods is the linear expert: a nonlinear law needs several cells whatever KK is.

Questions

1. Should "complete/exact law recovery" be revised, given the constant in LRR and PER?

We prefer to keep the rule and the term. The constant belongs to SRBench's symbolic-solution rule and a different rule would make our rates incomparable with published ones. At noise 0, to which the rates in the abstract refer, the constant almost never decides: of the 712 laws counted as recovered for BePart srbench-cap, 694 are the true law once constants are rounded, and the other 18 differ from it by a constant. Without the constant, PER would be 52.0% instead of 53.7%. Under noise the allowance is needed: an offset or a factor fitted to noisy targets rarely rounds to the true value.

2. Should the claim that all gating mechanisms are necessary be softened?

Yes, for the two mechanisms the reviewer names. The submitted paper did not claim that all are necessary, but Sec. 3 did not say which ones carry the effect, the revision does.

3. Do the experiments support conclusions about real-world applications?

Our claims on recovering regimes and laws are for static regime regression with known ground truth. On real-world data we test prediction accuracy only. Noise on the target is covered, however, up to 30% (App. E). At 10% the partition is still found (ARI 0.80 for srbench-cap) and exact laws are lost. Derivatives estimated from trajectories are not covered, and history-dependent switching cannot be expressed by a partition of the input space. The limitations now say so.

4. Does the 41% single-component rate indicate over-partitioning?

Yes. The selection only collapses cells whose predictions agree on the validation split, so a piecewise approximation that fits the validation data better than the best single expression survives. This becomes noticeable in symbolic recovery (at noise 0, 34% of the laws against 37% at k=1k=1), but not in accuracy (R2>0.999R^2 > 0.999 in 63% of the runs, the same as k=1k=1). Sec. 6 names it as the first thing to fix.

The revised paper is now uploaded, and the supplementary material and the anonymous repository are updated.

Weakness 1, identifiability: Sec. 2 states the two conditions and defines the canonical form (p. 3). App. B explains what is identifiable without them, why the canonical form is unique and why analytic laws are needed (pp. 26-27), and it describes the scoring on the test points (pp. 27-28). BePart runs without the absolute value on the regime suite (Sec. 5.1, p. 7). App. D.8 shows what the canonical form changes for every method (pp. 49-50, Tab. 23).

Weakness 2, fixed number of components: App. D.7 reports the four baselines with the true number of regimes and with KK selected on validation data (pp. 47-49).

Question 2: Sec. 3.4 says which mechanisms carry the effect (p. 6), and Sec. 5.3 reports the ablation of the repeated runs (pp. 8-9, Tab. 12 on p. 42).

Questions 3 and 4: the limitations state what the experiments do not cover, and they name the choice of the number of components on single-law problems as the first thing to fix (Sec. 6, p. 10). The robustness study with up to 30% noise is in App. E (pp. 52-53).

We would be grateful if you could take a look at the revised paper and let us know whether it addresses your concerns.

Reviewer_txDZOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

BEPART evolves symbolic gates and local equations to discover piecewise laws. The experiments evaluate both prediction and structural recovery, with strong results on clean data but much less reliable recovery under noise.

Critical strengths
  1. BEPART provides a coherent way to discover piecewise symbolic laws.
  2. The evaluation clearly shows when BEPART recovers the underlying structure and where it fails.
Critical weaknesses

W1: Some baseline comparisons mix recovery ability with the choice of model size. In the noiseless experiments, one baseline consistently uses too many regions to achieve complete recovery under the metric. W2: Conditional PySR's partition scores may penalize inactive conditions, but the effect on the reported runs is unknown. Evaluating every condition everywhere can create more regions than the expression actually uses.

Questions for authors
  1. How do the baseline recovery results change when K is selected on validation data?
  2. How do conditional PySR's ARI and PER change after merging cells that yield the same law?
2 comments

Thank you for the review. Both weaknesses could be checked, the first with an additional run of the baselines and the second on the submitted runs. Section, table and appendix numbers refer to the revised paper, which we will upload as soon as possible.

Weaknesses

1. Baseline comparisons mix recovery ability with the choice of model size

We agree. The linear model tree, PARC and the two mixtures ran with K=8K = 8 on every problem, while the problems have two to five regimes and PER requires exactly kk cells. The linear model tree is the clearest case: its trees have more leaves than regimes in all 380 fits at noise 0, so its PER is 0 by construction, although it recovers 21% of the laws (submitted Tab. 1). PARC has more cells than regimes in 370 fits. The revision changes two things. It adds the four baselines with KK selected on validation data. For every method, the revised metric (reply to Reviewer YE5K) merges cells with the same law before scoring, so that a model is not penalized for representing one regime by several cells.

2. Inactive conditions of PySR with conditionals

Yes, we agree that the effect is unknown. We read the partition off the expression by evaluating every condition at every test point. A condition that has no effect on the output in part of the input space still splits that part into two cells with the same law. The effect on the reported runs is small (see question 2). In the revision such cells are merged, as for every other method.

Questions

1. How do the baseline recovery results change when KK is selected on validation data?

They change little. We fitted the four baselines with K=1,…,8K = 1, \dots, 8 on 80% of the training set (the tree with 1, 2, 4 and 8 leaves), selected the KK with the lowest error on the remaining 20% and refitted with that KK on the whole training set. The table gives the results at noise 0 (K=8K = 8 | KK with lowest validation error). The numbers of this answer are over the 38 problems of the submission, so that they can be set against the submitted tables. The revised paper reports them over its 46 problems (8 were added, see our reply to Reviewer Q8Nj), so its values differ.

Methodmean KKARILRRPER
Linear model tree7.40.43 | 0.430.21 | 0.205.3% | 5.3%
PARC6.80.38 | 0.400.16 | 0.163.2% | 6.6%
Mixture of experts (linear)6.50.45 | 0.460.00 | 0.000% | 0%
Mixture of linear regressions (EM)5.10.35 | 0.330.17 | 0.135.3% | 5.3%
BePart default (k=k= auto)0.830.5032%
BePart srbench-cap (k=k= auto)0.930.7563%

Validation still leads to a KK that is too high, i.e. it is above the true number of regimes in 363 of the 380 fits of the tree, 354 of PARC, 369 of the mixture of experts and 286 of the EM mixture. Additional cells improve the piecewise-affine approximation of a nonlinear law and lower the validation error.

Additionally, we ran these baselines with KK set to the true number of regimes (for the tree, the smallest tree with at least that many leaves). Even with the true KK the baselines do not improve substantially: the ARI is at most 0.40, lower than with K=8K = 8 for all four, and PER at most 5.3%.

2. How do conditional PySR's ARI and PER change after merging cells that yield the same law?

On the submitted runs at noise 0, 17 of the 380 fits contain cells that yield the same law. Merging them changes the mean ARI from 0.634 to 0.638 and leaves PER at 17.4%. No fit becomes piecewise-exact, because in each of the 17 at least one law is still not recovered after merging. At 10% noise there are 11 such fits, the ARI stays at 0.56 and PER at 1.8%.

Too many cells are not the main obstacle for this baseline. Before merging, 41 fits at noise 0 have more cells than regimes and 109 have fewer, 55 of them without any condition. In 4 of the 17 fits that contain cells with the same law, the ARI goes down, because the expression has the same law on both sides of a condition that separates two true regimes.

The revised paper is now uploaded, and the supplementary material and the anonymous repository are updated.

Weakness 1 and question 1, the number of cells of the baselines: App. D.7 reports the four baselines with K=8K = 8, with KK selected on validation data, with the true number of regimes and with the best KK per fit (pp. 47-49, Tab. 20 to 22). The selection rule is in App. B (p. 29). Sec. 5.2 says that the fixed K=8K = 8 is not what limits these baselines (p. 8). The tables by problem property repeat the four baselines with the selected KK (Tab. 17 to 19, pp. 46-47).

Weakness 2 and question 2, inactive conditions of PySR with conditionals: cells with the same law are merged before scoring, for every method. App. B describes the scoring (p. 27) and how the cells of PySR with conditionals are read off its expression (p. 28). Tab. 23 counts per method how often a merge or a split changes the cells (p. 50).

The tables of the paper are over all 46 problems of the revised suite. The table in our reply is over the 38 problems of the submitted version, so its values differ slightly.

We would be grateful if you could take a look at the revised paper and let us know whether it addresses your concerns.

Reviewer_Q8NjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper proposes a new machine learning model that learns separate symbolic representations for different regions of the feature space. Moreover, the regions themselves are defined by symbolic expressions. To train the model, the authors propose an algorithm based on genetic programming and expectation0maximization approach. To evaluate the model, the authors propose a new set of synthetic benchmark problems. Some of them are inspired by physical systems or previous literature. The method is compared with other relevant approaches and evaluated across four different metrics.

Critical strengths
  • The paper addresses an interesting and often overlooked problem. These kinds of approaches allow symbolic regression to be applicable to more complicated settings where a single equation may not be sufficient to describe the dataset.
  • The attempt at making the routing algorithm/gates interpretable is worth noting.
  • The presentation is mostly clear and easy to follow. However, there are places where the wording sounds a bit too much LLM-generated and should be improved.
  • The authors contribute a new set of problems for benchmarking
  • The method is evaluated on four different metrics
  • The baselines chosen for comparison are appropriate
  • An ablation study is provided
Critical weaknesses

I think the biggest weakness of the paper is a limited discussion on the interpretability of the routing algorithm. This problem does not exist when there are only two regimes (as, for instance, shown in Figure 1), but nevertheless I find it to be an important limitation. The routing rule is argmax over several expressions. I would argue that usually this is not a very interpretable rule. To understand it properly, the user would need to analyze for which subsets of inputs a particular rule is chosen as having the maximum value. This may not be easy, as each symbolic expression may be quite complex rather than a simple linear equation. In my opinion, a much simpler way to describe a region of feature space is to use a simple inequality or a conjunction of 2 or 3 simple inequalities. In fact, all descriptions of regions in the proposed suites are exactly of this kind. Something that might be easily described by a decision tree. Thus, I find the criticism of other methods for using decision trees or simple thresholds for partitioning the space unwarranted. Not only do those methods give simple rules for routing, but they are also well-aligned with the proposed suite of problems and very likely with real-world scenarios.

To make it concrete, what I mean is: If we have four regimes described by the following gates. c1:x1×x2c_1: x_1 \times x_2

c2:sin⁡(x1)×x1+x2c_2: \sin(x_1)\times x_1 + x_2

c3cos⁡(x2)2+x1c_3 \cos(x_2)^2 + x_1

c4:x1x2+1c_4: \frac{x_1}{x_2+1}

Each gate is a simple expression, but the overall rule describing samples that will be assigned to region 1 looks like this:

$ x_1 \times x_2 > \sin(x_1)\times x_1 + x_2 \wedge x_1 \times x_2 > \cos(x_2)^2 + x_1 \wedge x_1 \times x_2 > \frac{x_1}{x_2+1}$

This no longer looks that simple. The paper needs to discuss it and, ideally, propose ways to address it.

The second weakness that impacts my assessment is that the paper sounds a bit too much like an LLM-written in certain places. I don't claim that the paper is entirely written by an LLM. However, certain phrases and sentences do sound awkward and are likely the result of using an LLM for polishing the language. This is present throughout the paper and will be difficult to address - I am aware of that. I don't think this should be the sole reason for rejection. Some things that sound awkward to me:

  • "Hyperparameters in the following are named where they matter,"
  • "The complexity objective is what keeps partitions honest:"
  • "and one that does reroute points changes the error by a jump."
  • "Random variation therefore finds a second regime occasionally and a fourth almost never"
  • "Two further ingredients keep the experts accurate without search effort."
  • "calibrated on their own cells the moment the child is born:"
  • "The remaining baselines lack expressiveness, not budget"
  • "Noise separates the two halves of the task"

This is not necessarily wrong, but I think the paper deserves a pass to polish such things.

Finally, the proposed suite seems a bit limited in terms of dd. 33/38 problems have d≤2d \leq 2. Also, most problems have k≤3k\leq 3 (35/38), but I think this is less of an issue.


Other minor things:

  • ARI (the abbreviation) is used early (in the abstract) without explanation. This is not a standard SR metric, so it would be good to describe what it is
Questions for authors

Clarification questions:

  • What is a "crowding distance" in l145-l146?
  • What does it mean to "change the error by a jump" in l154-l155?
  • Why is PS-Tree not evaluated for LRR and PER? The expert models are symbolic expressions.
  • In the ablation study, you say "Removing the grow chain or the merge proposals does not hurt,...". But I think the results show that it actually improves the performance. Could you comment on that?
5 comments

Thank you for the review. Section, table and appendix numbers refer to the revised paper, which we will upload as soon as possible.

Weaknesses

1. Interpretability of the routing rule

We agree that the routing rule is hard to read in its raw form, and the paper lacked this discussion. The cell of a component is where its gate exceeds every other gate, a conjunction of k−1k - 1 inequalities ci−cj>0c_i - c_j > 0. It can often be reduced to a simple form by simplifying each difference ci−cjc_i - c_j and omitting the inequalities that follow from the others. For example, with the gates x1x_1, x0+x1x_0 + x_1, 00 and x0x_0, the cell of the first gate is x1>x0+x1∧x1>0∧x1>x0x_1 > x_0 + x_1 \land x_1 > 0 \land x_1 > x_0. The first inequality simplifies to x0<0x_0 < 0, and the third follows from the other two, so the cell is x0<0∧x1>0x_0 < 0 \land x_1 > 0. An inequality with a single input is rewritten as a threshold: log⁡(x0)>0\log(x_0) > 0 becomes x0>1x_0 > 1. The library now creates this simplified form and provides it alongside the form returned by the search.

We counted how often the simplified rule is the true one on the 23 problems whose regimes are separated by thresholds (230 fits, srbench-cap, noise 0). BePart finds the true partition in 80% of the fits and writes it with thresholds only in 47% (71% with single-input inequalities rewritten as thresholds).

Tree-based methods always write thresholds, but they rarely recover the true ones. On the same problems the linear model tree has the true partition in 9% with K=8K = 8 and in 17% with KK matched to the true number of regimes, because its affine leaves cut a nonlinear regime into several cells. The criticism of trees rests on this, and Sec. 5.2 now states it.

The simple form is not always reached: inequalities that relate several inputs in another way stay hard to read (limitations, App. D.9).

2. Wording

We agree. The revision rewrites all eight sentences and others we found during the polishing pass.

3. Input dimension and number of regimes in the suite

We agree, and the revision adds 8 problems with 3 to 6 inputs, in which every input enters a boundary or a law (App. C). They are hard for every method. BePart (srbench-cap, noise 0) has an ARI of 0.75 on them, against 0.93 on the 38 problems of the submission, and recovers the complete piecewise law in 9% of the fits, against 63%. PySR with conditionals falls from 0.69 to 0.23 in ARI and from 23% to 8% in complete laws. For 6 of the 8 problems no method recovers the complete law in any fit. This cannot be attributed to the number of inputs alone, however, because the boundaries and laws of these problems are also more complicated than in most of the other problems. None of their boundaries is a threshold on one input, and most of their boundaries and laws are products or ratios of several inputs. App. D.6 now also compares the methods by group of problems (axis-alignment, number of inputs, number of regimes). BePart leads in ARI in every group at every noise level.

4. ARI used in the abstract without explanation

The abstract now writes out "adjusted Rand index" and explains its values.

Questions

1. What is the crowding distance (l. 145-146)?

It is the measure by which NSGA-II chooses among the members of one front. When a front is too big for the places left in the next population, its members are ranked by the distance between their two neighbours on the front, and those in crowded regions are thus dropped first. The revision explains this (Sec. 3.1, App. A.1).

2. What does "changes the error by a jump" mean (l. 154-155)?

As a function of a gate, the routed error is piecewise constant. A small change of a gate either reroutes no training point and leaves the error unchanged, or reroutes at least one and changes it by the loss difference between the new and the old expert at that point. The revision says so.

3. Why is PS-Tree not evaluated for LRR and PER?

The leaf models are symbolic, but the export of the library fails or prints placeholders for the constants. For the revision we re-ran PS-Tree and stored the predictions of every leaf model, which allows the accuracy-based check (CART reproduces 8%, adaptive reproduces 5%, BePart with srbench-cap 80%).

4. Does removing the grow chain or the merge proposals improve the performance?

Both are close to neutral in both versions. Without the grow chain the ARI changes by +0.017±0.007+0.017 \pm 0.007 in the submitted Tab. 12 and by −0.004±0.007-0.004 \pm 0.007 in the revision. Without the merge proposals it changes by −0.004±0.007-0.004 \pm 0.007 and −0.006±0.006-0.006 \pm 0.006. The small gain without the chain does not reappear because the search was run again with the same seeds and budgets, but with a primitive set that no longer includes the absolute value. On the same 38 problems and under the submitted scoring the new runs give 0.000±0.0070.000 \pm 0.007, so we read the submitted difference as a fluctuation of the search. Sec. 3 and Sec. 5.3 now state that neither has a measurable effect.

Thank you for your response. I believe the proposed changes would improve the paper, but I cannot find them in the available PDF. Have the revisions been uploaded yet? It is difficult to assess them otherwise.

I appreciate the new algorithm for simplifying the descriptions of the found regions. I would like to see how complex the functions are in practice on real-world black-box datasets (can be from SRBench). It would be helpful for readers to see how interpretable the models can be expected to be.

Thank you for the follow-up. The revision is not uploaded yet, and we apologise for the delay. We are still finalising it, but we plan to upload it tomorrow.

To show how complex the models are on real-world data, the revision adds a case study (App. D.10). It takes the 60 datasets of the SRBench black-box suite that are not synthetic Friedman problems, and for each the fit with the median test R2R^2 (srbench-cap, kk = auto). The median model has 117 nodes (quartiles 47 to 244), and 7 models are a single formula. The appendix prints the models at the 10th, 25th, 50th and 75th percentile of size, so the examples are not chosen by hand. The one at the 10th percentile, for electricity usage, is y^=−0.43sin⁡(2.42x0+2.42x1)−0.02+0.38e−x0\hat y = -0.43 \sin(2.42 x_0 + 2.42 x_1) - 0.02 + 0.38 e^{-x_0} for x0<0.11x_0 < 0.11 and y^=−0.88\hat y = -0.88 otherwise. The smaller models are easy to read in this way. Above the median, the rule or the experts contain terms without an obvious meaning and the appendix states that.

Comments on the paper

1 comment

The revised paper and its supplementary material are now uploaded, and the anonymous repository is updated. The following changes were made:

1. Identifiability and scoring. Sec. 2 states the conditions under which the regimes and their laws are identifiable and defines the canonical form (p. 3). App. B explains what is identifiable without these conditions, why the canonical form is unique, and how every model is scored in this form on the test points (pp. 26-28). App. D.8 shows what this changes for every method (pp. 49-50).

2. Regime suite and runs. The suite has 8 more problems with 3 to 6 inputs, 46 in total (App. C, pp. 29-36). We added them because only 5 of the 38 submitted problems had more than two inputs. BePart runs without the absolute value among its primitives on the suite (Sec. 5.1, p. 7), and all its runs there were repeated. All tables and figures of the regime suite are regenerated. The added problems are harder, so the recovery rates over the 46 problems are somewhat lower than over the 38, for BePart and for the strongest baseline alike. Without noise, BePart (srbench-cap) now reaches an ARI of 0.90 and recovers the complete piecewise law in 54% of the fits (Tab. 1, p. 8), against 0.93 and 63% on the 38 problems. BePart leads wherever it led in the submitted version (in ARI at every noise level, and in law recovery without noise).

3. Number of cells of the baselines. The four baselines with a fixed number of cells are also reported with the number selected on validation data and with the true number of regimes (App. D.7, pp. 47-49).

4. New analyses. App. D.6 gives the results by boundary type, number of inputs and number of regimes (pp. 45-47). App. D.9 writes the routing rules of the reported models in a simple form and counts how often they are the true rule (pp. 50-51). App. D.10 shows fitted models on real-world data (pp. 51-52).

5. Text. The introduction names the challenges of recovering piecewise laws (p. 1). Sec. 3 has an overview figure of the model and its search (Fig. 2, p. 4), explains the crowding distance (p. 4, in detail in App. A.1, p. 16) and says which mechanisms carry the effect (p. 6). The ablation is reported for the repeated runs (Sec. 5.3, pp. 8-9), and the limitations are extended (Sec. 6, p. 10).

We kindly ask the reviewers to take a look at the revised paper. We are happy to answer any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.