acceptodds
Under review as a conference paper at ICLR 2027

CANOPY: Closed-Form Neurosymbolic Composition with a Small State-Space Reducer

Abstract

Compositional reasoning benchmarks such as long-form LRA-ListOps and the DeepMind Mathematics arithmetic suite are tree-evaluation problems written as linear token streams. A model reading only the tokens must recover the tree structure, a language problem, and execute the operator rule at each node, a symbolic one. These carry different inductive biases, and end-to-end sequence models continue to trail systems that route inputs through an explicit composition rule. CANOPY separates the two at inference time. A parser supplies the tree, a small state-space reducer approximates each atomic operator call as a probability-mass function (PMF) over the answer, and a fixed closed-form composer propagates those distributions up the tree. Reducer and composer exchange nothing but PMFs, so either can be swapped or retrained without touching the other. A sub-1M-parameter reducer reaches 0.9535 on long-form LRA-ListOps, above the tree-induction band and above end-to-end baselines carrying up to 90x more parameters. On DM Math addition chains it reaches 0.989 against a 30M-parameter reference at  0.94. Mambino, the reducer instantiated here, carries an internal predictor branch that feeds its own one-step prediction error back into the recurrence. We partition every evaluation by whether an operator call was seen in training. Ablating the branch costs 2.8 pp or less on seen calls, and 25.0, 11.3 and 5.1 pp on unseen multiplication, addition chains and single-operation addition, respectively. On LRA-ListOps, whose operators are saturating, ablating it instead improves composition by 4.45 pp. The branch earns its parameters only where the reducer must compute rather than recall.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
compositional reasoning, neurosymbolic reasoning, state-space models, closed-form composition, probability-mass function propagation, tree evaluation, long-range reasoning, LRA-ListOps, DeepMind Mathematics, parameter efficiency, predictive coding, train-test overlap

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_rbxmOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies compositional benchmarks such as long-form LRA-ListOps and the arithmetic modules of DeepMind Mathematics, which the authors view as tree-evaluation problems written out as flat token sequences. The proposed framework, CANOPY, separates structure recovery from operator execution at inference time

Critical strengths

None

Critical weaknesses
  1. writing issues:
  • repeatedly emphasizes large headline numbers (e.g., "90x" speedups and "0.9xx" scores).
  • uses phrasing and notation that obscures meaning rather than clarifying it — for example, expressions like modules.<module>.train(ef) and abs(pred − float(true value)) appear without being properly defined or motivated
  1. Reproducibility concern, please ref to questions section.
Questions for authors
  1. Table captions: there are only 2 tables in the main text, so why do the captions take up so much space? (L270-280, L345-353) It only makes the reading experience worse. I'm also not sure it's appropriate to put so much explanation in the caption area — so many citations and parameters crammed in there.

  2. Meaningless citation of python code. What does build_atomic_reducer_dataset_v10.py even mean??? And also it doesn't show up in your repo (mambino-iclr-release-9018).

2 comments
AuthorsOpenReview ↗

Response to Reviewer rbxm

W1: repeated emphasis on headline numbers such as "90x".

We agree. "90x" is removed from the abstract. The abstract now states that the end-to-end baselines must also learn the tree.

W2: undefined notation.

We agree. modules.<module>.train(ef) and abs(pred - float(true value)) are removed. Section 5.2 and the Table 2 caption now say "training-generator questions at entropy setting (3, 10)", and the caption defines EM_0 as an exact value match.

Q1: table captions are too long.

We agree. The table captions are shortened. The details are in the text, and the LLM baseline protocol is in Appendix I.

W3 / Q2: reproducibility; build_atomic_reducer_dataset_v10.py.

We agree that a script name is not a description, and the script was missing from the anonymous release. Section 5.2 now describes the data instead. It is a directly sampled integer distribution of (op, a, b) tuples, seed 0, not drawn from the DM Math generator. The script is now in the anonymous repository, with a README entry.

Thanks, but i'd like to maintain my score

Reviewer_dTZnOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CANOPY, a neurosymbolic framework for compositional reasoning. The framework uses an external parser to recover expression trees, a small state-space neural reducer to predict probability distributions over atomic operator outputs, and a fixed closed-form composer to propagate these distributions through the tree. The authors also introduce Mambino, an SSM reducer augmented with a prediction-error feedback branch.The method is evaluated on LRA-ListOps and selected DeepMind Mathematics arithmetic tasks, reporting high accuracy with fewer than one million neural parameters. Additional experiments compare hard-argmax recursion with PMF propagation and investigate the contribution of the predictor branch on seen and unseen operator calls.

Critical strengths
  1. Clear modular design. The separation between parsing, neural reduction, and symbolic composition is conceptually simple and well presented. The PMF-based interface allows different reducer architectures to be used with the same composer, as demonstrated by the Tree-LSTM experiment.

  2. Useful empirical analysis. The paper provides detailed ablations of the predictor branch, including separate evaluations on seen and unseen operator calls. The authors also report negative results, such as the degradation caused by the predictor branch on ListOps and the superior performance of a zero-residual symbolic baseline for multiplication.

Critical weaknesses
  1. The problem motivation and the necessity of the learned neural component are not sufficiently established.

My main concern is whether the proposed framework addresses a sufficiently meaningful machine learning problem in the evaluated settings. Both benchmarks involve explicitly defined mathematical operators, and CANOPY already assumes access to the expression tree and the exact operator semantics through its parser and closed-form composer. Under these assumptions, a purely symbolic evaluator can directly compute the correct answer without learning the atomic operations.

This raises a fundamental question: why is a neural network needed to approximate deterministic operations such as addition, multiplication, MIN, or MAX when their exact computation rules are already available? In fact, Section 5.2 reports that removing the learned residual entirely achieves perfect accuracy in the evaluated multiplication setting.

I agree that combining learned primitives with symbolic composition can be valuable, particularly when the primitives are unknown or must be inferred from noisy observations. However, the current experiments do not demonstrate such a setting. Consequently, the strong benchmark results do not clearly establish what additional capability the learned reducer provides beyond conventional symbolic computation.

  1. The experimental comparisons do not adequately isolate the contribution of the proposed method.

CANOPY is compared against end-to-end sequence models that must learn the compositional structure and operator behavior from raw tokens, whereas CANOPY receives an externally parsed tree and uses manually specified composition rules. These systems therefore solve substantially different problems, making the large performance and parameter-efficiency improvements difficult to interpret as evidence of a stronger neural architecture.

There are also concerns about the DeepMind Mathematics evaluation protocol. Section 5.2 uses heavily filtered integer-operand populations, with an acceptance rate of only 2.8% for the multiplication module. Furthermore, Table 2 evaluates examples generated by modules.train(ef), whereas the reference results in Saxton et al. (2019), Figure 4, are reported on the benchmark's interpolation test distribution. The official dataset implementation uses different entropy configurations for these two settings. Therefore, even the claimed population-matched comparison on add_sub_multiple does not appear to use an identical evaluation distribution.

Overall, the experiments demonstrate that the proposed hybrid system performs well on restricted deterministic reasoning tasks, but they do not sufficiently establish its advantage over the most relevant symbolic alternatives or support direct comparisons with end-to-end neural baselines.

Questions for authors
  1. What is the intended problem setting in which learning the atomic reducer is necessary? Given that both the expression structure and the operator semantics are known, a purely symbolic evaluator should solve the evaluated tasks exactly. What capability does CANOPY provide that such an evaluator cannot? Is there evidence supporting the framework's usefulness when the primitive operations cannot be directly implemented symbolically?

  2. Can the authors clarify the comparison protocol for DeepMind Mathematics? In particular, why are samples drawn from modules.train(ef) rather than the official interpolation test generator used in the reference evaluation? How would the results change under an identical evaluation distribution?

  3. How much of the improvement comes specifically from the prediction-error mechanism? The current predictor-branch ablation also changes model capacity. Would a parameter-matched two-branch SSM without prediction-error feedback achieve similar improvements on unseen arithmetic operations?

1 comment
AuthorsOpenReview ↗

Response to Reviewer dTZn

W2: CANOPY gets a parsed tree and manual composition rules, so comparisons do not isolate the contribution.

We agree that the end-to-end baselines solve a harder problem, and the revised abstract says so. We added two experiments that hold the given structure fixed and vary what is learned.

First, the tree alone does not explain the gain. A Tree-LSTM given the gold LRA tree and trained on the root label reaches 0.5905. With the same tree, CANOPY's composer over per-call PMFs reaches 0.9205, 0.9535, and 0.989 with Tree-LSTM, Mambino, and Transformer reducers. This isolates the composition layer: given a tree, exact PMF composition over learned atoms separates 0.59 from 0.92-0.99.

Second, the composer's contribution depends on the atoms in a measurable way. On LRA, where atoms are imperfect, Algorithm 2 improves all three reducers by 2.1 to 10.2 pp over Algorithm 1. On DM Math, where atoms are near-exact, Algorithm 2 is only as good as the reducer's calibration (Q3 below).

The composition rules are hand-written by design. They cover operators whose semantics are known. Section 7 covers operators whose semantics are not known: hidden rules, image operands, and training on final answers only.

W1 / Q1: the setting in which learning the reducer is necessary.

We agree with the premise. When the structure and operator semantics are known, a symbolic evaluator solves these tasks exactly. The setting we intend is the one you describe: primitives that are unknown or inferred from noisy observations. Section 7 tests it, with hidden rules, MNIST operands, and final-answer training. The reducer reaches 0.881 and 0.856, against 0.205 and 0.002 for a lookup table (Table 3). The closed-form composer still applies above these learned atoms in mixed trees (Figure 5b). The 1.000 of the delta = 0 baseline (Section 5.2, Appendix K) is consistent with this view, and Section 1 now says so.

W3 / Q2: DM Math protocol.

The filter keeps integer-operand questions, since the reducer works on integers. Acceptance is 2.8% on mul, 4.8% on add_or_sub, and 97.6% on add_sub_multiple (Section 5.2). We re-ran add_sub_multiple on the official interpolation generator with the same checkpoint, filter, n = 1000, and seed. Under Algorithm 1 it scores 0.994 on the training generator and 0.995 on the interpolation generator. Under Algorithm 2 it scores 0.986 and 0.987. So this comparison does not depend on the generator. On the interpolation generator, mul and add_or_sub questions use decimals and operands beyond the reducer's integer range, so those rows cannot be compared there. Section 5.2 states this.

Q3: a parameter-matched control without prediction-error feedback.

We ran it: a Mambino without the predictor branch, widened to 515,570 parameters (full model: 515,623), with the same data and schedule. On calls seen in training, the two are within 0.7 pp. On unseen calls (n = 487, 139, and 312), the branch adds 4.9 pp on add_or_sub and 15.1 pp on add_sub_multiple, and costs 9.3 pp on mul (0.753 against 0.846). The clearest effect is under Algorithm 2. On add_sub_multiple, the full model scores 0.986 and the matched control 0.824, while both score about 0.98 under Algorithm 1. A parameter-matched Transformer (one seed) shows the same pattern: 1.000 under Algorithm 1 and 0.835 under Algorithm 2. Measured directly, the full model leaves 0.5% of its mass off the true answer on correctly computed atoms. The matched control leaves 17.0% (expected calibration error 0.006 against 0.160). This mass compounds with depth. Four levels up the tree, the probability on the true value is 0.93 for the full model and 0.29 for the matched control. Accuracy there is 0.985 against 0.764. So the branch mainly improves calibration, which makes the reducer's PMFs composable. Its effect on argmax accuracy depends on the operator. Table 2 now shows both in the main text, and Section 6.2 discusses them.

Reviewer_ua12OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CANOPY, a parse-then-compose framework for tree-evaluation benchmarks. A supplied parser recovers the operator tree, and a small diagonal-scan SSM reducer (Mambino) outputs a PMF at each atomic-reduction node. A fixed closed-form composer then combines these PMFs up the tree (Algorithm 2), or uses the argmax value at each node (Algorithm 1). Mambino adds a predictor branch that feeds its prediction error (ε\varepsilon-injection) back into the main SSM. The paper reports 0.9535 on long-form LRA-ListOps with a 110K-parameter reducer, and 0.989 on DM Math (add_sub_multiple).

Critical strengths
  1. The design is simple but works. Algorithm 2 is 8.0% better than Algorithm 1 with the same reducer. A Tree-LSTM reducer of similar size reaches 0.9260 with the same composer. Both results support PMF-layer decoupling.
  2. The results are strong for the model size. With a 110K-parameter reducer, CANOPY reaches 0.9535 on long-form LRA-ListOps. This is well above the tree-induction models (best 0.801) and the end-to-end SSMs (about 0.62–0.64).
Critical weaknesses
  1. The novelty is unclear. Each part of the method has been done before:
  • DeepProbLog report 98.7% on ListOps with a TreeLSTM given the gold parse tree.
  • DeepProbLog and CTSketch combine neural outputs with exact probabilistic inference. CTSketch also passes distributions through a tree of sub-programs.
  • Neural Execution Engines and this paper learn arithmetic primitives and use them inside symbolic algorithms.
  1. The inputs at atomic-reduction nodes are plain numbers, and the closed-form rules are exact on one-hot inputs (5000/5000). So a fully symbolic evaluation should get 100% on both benchmarks. Does this mean the neural reducer may not be needed? The authors do not test any task where the primitive cannot be computed by rules, such as noisy or image inputs.
  2. CANOPY gets the parse tree and labels for every operator call. The baselines only get the final label. The parser is also not counted in the parameter numbers. So the claim of “up to 90× fewer parameters” does not show a real advantage of the method.
  3. With δ≡0\delta \equiv 0, the result is the same on mul (1.000) and better on MUL chains (1.000 vs. 0.9095). The authors still list Mambino-Residual as a contribution.
Questions for authors
  1. How is CANOPY different from a gold-tree TreeLSTM, CTSketch, or DeepProbLog when given the same supervision?
  2. What is the accuracy of a fully symbolic evaluation? Can you test a task where the leaves cannot be computed by rules, for example image inputs?
  3. Can you add a small Transformer reducer with a similar parameter count? Appendix E says it should work. Can you also add baselines that use the given tree but only the final label?
  4. Can you explain the contribution of Mambino-Residual?
1 comment
AuthorsOpenReview ↗

Response to Reviewer ua12

W1 / Q1: novelty against DeepProbLog, CTSketch, and NEE; how CANOPY differs from a gold-tree TreeLSTM given the same supervision.

CANOPY belongs to the family of neural outputs under exact symbolic inference. We do not claim the family. The difference is in what each system must be given. DeepProbLog and CTSketch need each operator's program, as logic or as a sub-program. NEE learns primitives from supervision on the intermediate steps of a given algorithm. In Section 7, the operator rule is not given in any form. The reducer learns it from final answers on image operands, and the closed-form composer works above those learned atoms. The composer also propagates PMFs through order statistics and multi-digit multiplication in closed form, with no knowledge compilation. For the gold-tree TreeLSTM, we ran the comparison you ask for (Q3b below). Given the gold tree but only the root label, it reaches 0.5905 on LRA-ListOps. CANOPY reaches 0.92-0.99 with three reducer families. The 98.7% figure is from Nangia and Bowman (2018) on the original ListOps, not the long-form LRA split. The revision cites it in Section 2.

W2 / Q2: fully symbolic accuracy; a task where leaves cannot be computed by rules.

A fully symbolic evaluation scores 1.000 on both benchmarks, since the closed-form rules are bit-exact on one-hot inputs (5000/5000). Following your suggestion, the new Section 7 uses image operands with hidden operator rules, trained on final answers alone. The reducer reaches 0.881 on hidden visual ListOps and 0.856 on hidden DM Math atoms. A lookup table over a digit classifier reaches 0.205 and 0.002 (Table 3).

W3: parse tree and per-call labels against final-label baselines; uncounted parser; "90x".

We agree. The parser is a deterministic routine with no learned parameters, and our counts include learned parameters only. The baselines learn the tree from tokens, so the parameter ratio is not a fair measure of architectural advantage. The revised abstract says so and drops "90x". In Section 7, the reducer is trained on final answers alone.

W4 / Q4: Mambino-Residual with delta = 0.

We agree. delta = 0 matches or beats the learned residual on both multiplication tasks. Mambino-Residual is removed from the contributions. Section 5.2 states this in one paragraph, and Appendix K gives the details.

Q3a: a small Transformer reducer.

We trained a two-layer Transformer reducer (108,250 parameters, against 110,192 for Mambino) on the same LRA atoms and curriculum, with three seeds. Under the same parser and composer, it reaches 0.968 (range 0.950-0.978) under Algorithm 1 and 0.989 (0.986-0.995) under Algorithm 2. Mambino reaches 0.8735 and 0.9535. This supports the central claim: the composer is reducer-agnostic, and three reducer families reach 0.92-0.99. We also trained a parameter-matched Transformer (0.51M) on DM Math. It computes every atom correctly, so Algorithm 1 scores 1.000 on add_sub_multiple. Algorithm 2 scores 0.835, against 0.986 for Mambino, because its PMFs are less calibrated (Section 6.2). So the reducer choice matters for PMF propagation, not for whether the composer applies.

Q3b: baselines that use the given tree but only the final label.

We trained an N-ary Tree-LSTM (578,314 parameters) on the gold LRA trees, supervised on the root label only. It reaches 0.5905 on the full test set. CANOPY reaches 0.9535 under Algorithm 2. Given the tree, a learned composition trained on final labels stays near the end-to-end models. The per-call PMFs and the closed-form composer carry the gain (Table 1).

Reviewer_roPJOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CANOPY evaluates tree-structured tasks in three parts: (i) a parser to get the tree, (ii) a statespacemodel predicts a PMF over the answer, and (iii) a hand-written closed-form composer propagates those PMFs up the tree. The authors provide promising results substantially outperforming existing approaches (not using SSMs), specifically give the methods param size.

Critical strengths
  • paper is interesting and relevant topic
  • documentation seems thorough, authors report negative results, which is a plus.
Critical weaknesses
  • the headline comparisons arent super fair given that the approaches are very different. Its still valid to do so, but its unclear which component now gives which boost.
Questions for authors
  • can the authors better ablate the single components of the approach?
1 comment
AuthorsOpenReview ↗

Response to Reviewer roPJ

W / Q: "It is unclear which component gives which boost. Can the authors better ablate the single components?"

We agree. The revision now varies each component with the others held fixed.

Component variedHeld fixedResult
Composer: Alg. 1 vs Alg. 2parser, reducerLRA: +8.0 pp (Mambino), +10.2 (Tree-LSTM), +2.1 (Transformer)
Reducer: Tree-LSTM, Mambino, Transformerparser, composerLRA, Alg. 2: 0.9205, 0.9535, 0.989 (3 seeds)
Composer removed: Tree-LSTM on the gold tree, root label onlyparser (gold tree)LRA: 0.5905
Reducer removed: symbolic leavesparser, composer1.000 (bit-exact)
Predictor branch, parameter-matchedparser, composer, sizeDM Math add_sub_multiple, Alg. 2: 0.986 with, 0.824 without

Read together: the tree alone does not carry the gain (0.59). The composer improves every reducer. The reducer family moves the result within 0.92-0.99. On DM Math, every reducer computes the atoms almost exactly, so Algorithm 2 depends on calibration. A Transformer reducer scores 1.000 under Algorithm 1 but 0.835 under Algorithm 2; Mambino scores 0.994 and 0.986 (Section 6.2). Section 7 adds the case with no rule at all. There the reducer is necessary, and a two-branch LeNet-5 reducer can fill the same slot (Figure 5b).

The end-to-end baselines must also learn the tree, which our parser supplies. The revised abstract states this. We present those comparisons as context, not as a like-for-like test.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions and response to all reviewers

We thank the reviewers for their careful reading. Three concerns are shared: (i) what is new (ua12, dTZn), (ii) whether a learned reducer is needed when the operator rules are known (ua12, dTZn), and (iii) which component gives which gain (roPJ, ua12, dTZn). The revision and four new experiments address all three.

What CANOPY is. A parser supplies the tree. A learned reducer outputs a PMF at each atomic call. A closed-form composer combines those PMFs exactly up the tree. The reducer and the composer share only PMFs. The contribution is this composition layer, not any one reducer.

The reducer is swappable, and the composer carries the gain. All rows use one parser and one composer on LRA-ListOps (full test trees, gold label):

ReducerParamsAlg. 1 (argmax)Alg. 2 (PMF)
Tree-LSTM0.11M0.81850.9205
Mambino0.11M0.87350.9535
Transformer, 3 seeds (new)0.11M0.968 (0.950-0.978)0.989 (0.986-0.995)
Tree-LSTM on the gold tree, root label only (new)0.58M-0.5905

All three reducer families reach 0.92-0.99. PMF propagation improves each one, by 2.1 to 10.2 pp. Given the same gold tree but no composer, a learned composition reaches 0.59. So the gain comes from exact composition over per-call PMFs, not from the tree alone.

When PMF propagation helps, and what it needs. On DM Math add_sub_multiple (n = 1000), a parameter-matched Transformer reducer (0.51M, one seed, new) computes every atom correctly. Its Algorithm 1 score is 1.000, but its Algorithm 2 score is 0.835, because its PMFs put mass on wrong values. Mambino scores 0.994 under Algorithm 1 and 0.986 under Algorithm 2. Removing Mambino's predictor branch at matched size (new) gives the Transformer's pattern: 0.981 and 0.824. We measured the cause directly. On correctly computed atoms, Mambino leaves 0.5% of its mass off the true answer. The matched control leaves 17.0% and the Transformer 15.3% (expected calibration error 0.006, 0.160, and 0.155). This mass compounds up the tree. Four levels above the atoms, the probability on the true value is 0.93 for Mambino and 0.29-0.30 for the other two. Accuracy at those nodes is 0.985 against 0.76-0.77. So Algorithm 2 recovers reducer errors when atoms are imperfect (LRA). When atoms are exact (DM Math), it is only as good as the reducer's calibration. The predictor branch makes Mambino's PMFs composable; it does not make its argmax more accurate (Section 6.2).

A setting where the reducer is necessary. A symbolic evaluator solves every atomic call whose rule is known and whose operands are tokens. Our composer rules are bit-exact on one-hot inputs (5000/5000), so it scores 1.000 on both benchmarks. The new Section 7 tests the case with no rule. The operator rules are hidden, every operand is an MNIST image, and the reducer is trained on final answers alone, with no digit labels. It reaches 0.881 on hidden visual ListOps and 0.856 on held-out hidden DM Math atoms (mean of three seeds). A digit classifier with a lookup table reaches 0.205 and 0.002 (Table 3). In mixed trees, the closed-form composer works above these learned atoms. It reaches 0.68-0.73 at 256 leaves, against 0.50-0.55 when each level re-reads an image (Figure 5b).

A correction. While preparing this response, we found an error in the Tree-LSTM row of Table 1. It had been scored against the parser's value on test trees truncated at 2,048 tokens, not against the gold label. Re-scored on full test trees against the gold label, it reaches 0.8185 under Algorithm 1 and 0.9205 under Algorithm 2 (before: 0.7445 and 0.9260). The revision uses the corrected row.

Other changes. The abstract now states that the end-to-end baselines must also learn the tree, and it drops "90x". Mambino-Residual is no longer listed as a contribution. Section 6 opens by stating what each ablation varies. Table captions are shortened.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.