acceptodds
Under review as a conference paper at ICLR 2027

Mitigating Reasoning Shortcuts in Abductive Learning with Semantic-Layer Initialization

Abstract

Abductive Learning (ABL) is a promising framework for integrating machine learning with logical reasoning. However, its generalization can be undermined by reasoning shortcuts, where seemingly strong reasoning performance masks unintended meanings of intermediate concepts. We trace this vulnerability to the learning–reasoning interface in ABL: associating input patterns with reasoning concepts is entangled with representation learning, leaving semantic alignment to random initialization and training dynamics. Meanwhile, the prevailing weak-binding practice treats inputs independently during reasoning, weakening cross-example constraints on what a predicted label means. To address these problems, we reformulate ABL with an explicit semantic layer that decouples semantic alignment from representation learning while enabling strong binding across related inputs. Under idealized perfect strong binding, we formalize shortcuts as concept transformations preserving reasoning outcomes and distinguish benign permutations from malignant non-injective collapses, yielding an identifiability limit up to benign equivalence. Guided by this analysis, we propose BASIL (Binding-Alignment for Semantic Initialization via Logic-consistency), a two-stage initialization. Stage I clusters inputs for strong binding; Stage II performs training-free semantic alignment by maximizing logic consistency. Experiments on visual and symbolic tasks show that BASIL substantially improves ABL's robustness to reasoning shortcuts.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Abductive Learning, Reasoning Shortcut, Neuro-Symbolic Learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_aaWcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a two-stage initialization method to improve the semantic alignment of learned intermediate concepts in Abductive Learning (ABL). In particular, the BASIL (Binding-Alignment for Semantic Initialization via Logic-consistency) approach reformulates ABL through an explicit semantic alignment comprising two parts: one abstraction — mapping inputs to logical constants — and one interpretation, which assigns pseudo-labels to concept predicates. More in details, BASIL carries out the following initialization stages:

  • Stage I: trains a contrastive clustering model and assigns one logical constant per cluster ("strong binding");
  • Stage II: searches for a bijective cluster-to-predicate mapping (interpretation) maximizing logic consistency with the underlying knowledge base. The search uses Gumbel-perturbed proposals and cross-entropy updates, followed by a final decoding step to assign pseudo-labels to the concept predicates.

The aligned initialisation can then be used to support learning in different abductive frameworks, including the original ABL, A3BL or DeepStochLog. Under an idealised "perfect strong binding" assumption in which the inputs are perfectly clustered as the ground truths, a Reasoning Shortcut (RS) problem can be expressed as a mapping between concepts that preserve the reasoning result. Bijective mappings define shortcuts called benign, and non-injective ones collapsing multiple concepts into one are called malignant.

The main theoretical results cover score separation under cluster errors (Theorem 2) and the correctness of the final decoding (Theorem 3). Experiments cover four CIFAR-10/MNIST arithmetic tasks (Addition, Decipher, Addition-Half and Addition-EvenOdd) and one synthetic symbolic task (Large-K). The results show that BASIL consistently improves the recovery of intermediate concepts with the intended semantics, up to benign relabelling, according to the reported metrics.

Critical strengths

The paper addresses a relevant and overlooked problem in the field of weakly supervised neuro-symbolic learning and it does so in a principled way. In particular, separating the abstraction from the interpretation in the initialization stage models explicitly where the ambiguity enters the learning process, which is an interesting perspective per se. I found the proposed method to be well motivated and structured: it plugs into the existing ABL framework easily without disrupting the original learning procedure. The theoretical results are sound and the proofs are clear. The baselines selected for the experiments are also appropriate, as the ablations are effective in isolating the contribution of each stage of the initialization on the full model, along with the comparison with the pretrained models.

Critical weaknesses

Please consider the following points.

  • W1) Defining bijective shortcuts "benign" goes beyond the identifiability framework the paper cites. The effect compounds over the experimental analysis.

The paper treats any bijection on concept values preserving the knowledge labels as a benign shortcut and counts the recovery of the intended semantics up to a mapping as a success. However, [1] defines intended semantics in a different way. Their Definition 3.3 allows a permutation of concept variables and element-wise invertible relabellings of their values, but only when the inference layer undoes them. This is what guarantees ultimately the correct reasoning result. When the knowledge is fixed, as in the current formulation of the framework, the identity is the only intended solution. Also, the shift-by-five on the Addition-EvenOdd task preserves only the training support, confirming that the intended semantics is not recovered as the framework promises: as stated below Table 11, BASIL recovers the intended semantics 10/20 times under identity only evaluation. Please report OOD RA, CA and Success under identity only, using a single relabelling per run. This leads to the second weakness, that is a problem of how realistic the evaluation is.

  • W2) The evaluation requires knowing in advance the reasoning shortcuts.

The evaluation for the core metrics 'Success', 'CA' and 'SSV' rests on the best case scenario defined over the set of benign shortcuts. For every task, the authors derive this set by exact search (App. D.1) or by hand (App. E.3). For a new task the set is unknown, it may be very large or impractical to compute in advance. On Kand-Logic from [2], for example, every permutation of colours, every permutation of shapes, and the shape↔colour swap would all count as benign. Even when the set is known, choosing among benign interpretations requires labels. For example, on Addition-EvenOdd, the protocol (App. E.1, l.1193–1198; l.348–349) applies both identity and shift-by-five to each run and keeps whichever scores higher against held-out sums and concept labels, separately for each metric. This is an oracle choice that a deployed model cannot make, and it is what turns 10/20 into the 20/20 reported in Fig. 2.

  • W3) The experiments put little pressure on BASIL in terms of the shortcut reasoning.

Three out of the five tasks proposed do not admit malignant reasoning shortcuts. This is unexpected for a paper whose main contribution is reasoning shortcut mitigation. Also, while Addition-Half does admit a malignant collapse, Stage II excludes it by construction (bijective interpretation with k = |P|): collapse is prevented by a hard constraint and not avoided by learning. Whether it re-emerges downstream is barely tested: visual ABL training is short, and the failing DSL+BASIL-init run on Addition-Half (19/20, SSV 10.8 ± 19.9, Table 10) is not analysed. Conversely, the challenging tasks proposed in [2] like Kand-Logic and BDD-OIA, highly favour the use of few concepts only through a knowledge signal that is not only ambiguous but also prone to reintroduce the collapse the initialization is trying to avoid. The open question is whether downstream ABL training re-introduces collapse when the knowledge strongly favours it, and this is not tested. This is fundamental to prove that the proposed initialization is effective in practice, as the paper claims.

  • W4) No in-the-wild, real-world evaluation is provided.

All visual tasks render digit concepts with MNIST or CIFAR-10 images, which are small, object-centred and low-resolution. It would noticeably strengthen the paper's claims if the authors provided a real-world, in-the-wild evaluation over a task prone to shortcut reasoning. I suggest the BDD-OIA task from [2] and, if not applicable, please explain which assumption prevents using BASIL for it.

  • W5) The framework is formulated in a limited way and the consequences are not discussed.

The predictor assigns each input exactly one of k mutually exclusive unary predicates. However, this excludes objects with several attributes (such as shape and colour in Kand-Logic or multi-label concepts such as in BDD-OIA) and relational concepts learned from perception. Only the single-object restriction is acknowledged and the other consequences are not discussed.

  • W6) The assumptions of Theorem 1 are stronger than intended and they do not match the experiments.

The second condition in Theorem 1 requires π\pi to preserve the feasibility of all the candidate groundings. This is stronger than what Definition 2 asks for and it fails to hold for shift-by-five on Addition-EvenOdd. For example, given the label 6, the pair (2,4) can be shifted to (7,9) and yet the sum 16 is not feasible. So Theorem 1 does not currently cover Addition-EvenOdd. For the same reason, calling these conditions "mild" is misleading. Finally, the core assumptions of the Theorem should be stated in the main text, as they are central to the paper's claims.

  • W7) Stage I operates under the hidden assumption of knowing the class of invariances that are relevant to the task.

Stage I also builds in assumptions about which variations are irrelevant, through its augmentations. The CIFAR pipeline (Table 4) includes colour jitter and grayscale conversion, which would remove colour as a concept in tasks where colour matters.

  • W8) The method relies on several hyperparameters and the paper does not provide a clear guidance on how to set them.

Stage II has τg\tau_g, ρ\rho, λ\lambda, η\eta, MM and TT, on top of the Stage I settings. The sensitivity study reported in Figure 5 varies each parameter by only about ±10−20%\pm 10-20\% around its default value, and τg=1.1\tau_g = 1.1 appears to cause failures already (16/20 on Large-K). This is particularly concerning for the cases of real-world tasks with in-the-wild data, where the tuning of hyperparameters is not trivial. Also the paper does not explain how the defaults are chosen in the first place.

  • W9) There is no systematic count of the number of reasoning shortcuts admitted by each task.

App. E.3 characterises shortcuts task by task, by hand. The central claim is that strong binding reduces ambiguity, and this would be much better supported by counting shortcuts. This is now standard practice in the RS literature, cf. [3, 4].

  • W10) Presentation, positioning and related work are not polished enough.

The introduction to reasoning shortcuts is brief, and a recent survey [4] on the topic is missing. Strong binding is also closely related to cluster-separability assumptions in [5] and the similarity-based consistency in ABL [6]: these connections should be adequately discussed.

Overall, the proposed method is well structured in spirit, but there are major weaknesses (W1-W6) which need to be addressed before the paper meets the ICLR expected standards. I am open to discussing the points above with the authors and I am happy to raise my score if they can be addressed in a revision.

References

[1] S. Bortolotti, E. Marconato, P. Morettin, A. Passerini, S. Teso. Shortcuts and Identifiability in Concept-based Models from a Neuro-Symbolic Lens. NeurIPS 2025.

[2] S. Bortolotti, E. Marconato, T. Carraro, P. Morettin, E. van Krieken, A. Vergari, S. Teso, A. Passerini. A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts (rsbench). NeurIPS 2024, Datasets and Benchmarks Track.

[3] E. Marconato, S. Teso, A. Vergari, A. Passerini. Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts. NeurIPS 2023.

[4] E. Marconato, S. Bortolotti, E. van Krieken, P. Morettin, E. Umili, A. Vergari, E. Tsamoura, A. Passerini, S. Teso. Symbol Grounding in Neuro-Symbolic AI: A Gentle Introduction to Reasoning Shortcuts. arXiv:2510.14538, 2025.

[5] L. Andolfi, E. Giunchiglia. Right for the Right Reasons: Avoiding Reasoning Shortcuts via Prototypical Neurosymbolic AI. NeurIPS 2025.

[6] Y.-X. Huang, W.-Z. Dai, L.-W. Cai, S. H. Muggleton, Y. Jiang. Fast Abductive Learning by Similarity-based Consistency Optimization. NeurIPS 2021.

Questions for authors
  1. Can the framework express Kand-Logic? If so, how would you construct the clusters, how many bijections would be benign under your definition, and how would Stage I augmentations handle colour as a concept?

  2. What happens if kk is unknown?

3 comments
AuthorsOpenReview ↗

Response to Reviewer aaWc (1/3)

We thank Reviewer aaWc for recognizing that we address an overlooked problem in weakly supervised neuro-symbolic learning and that separating abstraction from interpretation makes the source of ambiguity explicit. We also appreciate your assessment of the clarity of the proofs and of the ablations that isolate each stage's contribution.

Summary of the changes.

  • Evaluation (A1–A2). Tables 13 and 14 distinguish reasoning with the original digit names from concept recovery under one fixed relabeling.
  • Theory (A3). Section 3 and Appendix B state what the EvenOdd observations can identify and the additional conditions needed for identical ABL trajectory distributions.
  • Downstream learning and scope (A4–A9). The revision follows the same models from initialization to the final evaluation, tests both attributes on Kand-Logic, and gives exact shortcut counts in Table 8.

New or rewritten main-text passages and the titles of new appendix subsections are marked in blue.


Q1 (Weaknesses 1 and 2). What does Figure 2's 20/20 mean on EvenOdd, and what are the OOD results under the original digit names?

A1. For each learner, BASIL preserved all ten digit distinctions in 20/20 runs and recovered the original names in 10/20. All three improved on unseen sums under the original names. Figure 2 requires each digit's most frequent prediction to be correct under either identity or a global shift by five modulo ten. Both namings preserve all 16 training sums.

Original-name evaluation. Table 13 uses the identity for every example and metric. RA is reasoning accuracy, CA is digit accuracy, and SSV is the largest fraction of any digit predicted as one wrong digit (mean ± standard deviation over 20 runs).

LearnerInitializationSuccessRA (%)CA (%)SSV (%)
ABLPretraining0/205.2 ± 3.98.9 ± 11.198.7 ± 1.0
ABLBASIL-init10/2053.8 ± 22.745.3 ± 42.157.2 ± 40.3
A³BLPretraining0/207.8 ± 5.20.9 ± 0.499.1 ± 1.1
A³BLBASIL-init10/2056.4 ± 23.546.1 ± 43.255.8 ± 42.1
DeepStochLogPretraining0/207.4 ± 4.00.9 ± 0.599.2 ± 0.9
DeepStochLogBASIL-init10/2056.6 ± 23.746.1 ± 43.356.0 ± 41.6

ABL's raw OOD reasoning accuracy rises from 5.2% to 53.8%. Definition 3.3 of Bortolotti et al. allows permutations of concept variables and invertible relabelings of their values when the inference map compensates. Our rule is fixed. “Benign” means the bijection preserves concept distinctions and training outcomes. The EvenOdd shift can change unseen sums, which the original-name evaluation measures (Section 3). Bortolotti et al., Definition 3.3


Q2 (Weakness 2). Does BASIL need concept labels or the shortcut set? Does using one map per run change the OOD results?

A2. Stage II needs neither concept labels nor the shortcut set. For DeepStochLog with BASIL-init, fixing one map reproduced all four aligned metrics in every run. The original evaluation chose a naming per metric using held-out labels. In the replay, concept accuracy on the existing in-distribution split determined one map for all OOD predictions.

DeepStochLog obtained 20/20 successes, 80.6% RA, 89.6% CA and 13.7% SSV (Table 14). The same replay for A³BL gave 20/20, 89.4% CA and 14.1% SSV. Its saved outputs lack operand pairings for shifted RA, and the original ABL cohort lacks the per-example predictions for replay. Table 13 gives uncalibrated reasoning performance for all three.

Even the offline Success metric needs at most one candidate check given a labeled confusion matrix covering every class. Each row must have a unique most frequent prediction, distinct across rows. These determine the sole candidate, whose admissibility still has to be verified on the task support.

AuthorsOpenReview ↗

Response to Reviewer aaWc (2/3)

Q3 (Weakness 6). Does Theorem 1 cover the EvenOdd shift? Which assumptions are needed?

A3. Theorem 1 does not cover the shift. The identifiability limit on EvenOdd holds independently of that theorem. As you point out, (2,4)(2,4) is a feasible correction for sum 6 and its image (7,9)(7,9) sums to 16. Section 3 now states the conditions for equality in distribution of the ABL trajectories. The runs share the abstraction and initial states. Relabeling must preserve correction loss and every candidate's feasibility, abduction including tie-breaking must respect it, and updates must be independent of predicate names.

The two namings give identical training observations. With no side information distinguishing the two worlds, any learner's probability of recovering the original names averages at most one-half across them (Proposition B.1). The namings disagree on 50 of 84 unseen pairs. On a uniform draw from those pairs, any predictor incurs expected error of at least 25/84 in at least one world (B.2). For the trajectory question, B.3 shows that shared updates give identical trajectory laws if the correction-target distributions agree at the states training actually visits.


Q4 (Weakness 3). Does downstream training undo the initialization? What happened in the unsuccessful DeepStochLog run?

A4. On the two original tasks that admit exact concept mergers, every initially successful ABL run was also successful after downstream training. The single DeepStochLog failure started with two digits already confused.

For ABL on Addition-Half, 18/18 initially aligned runs retained all five digits on the ID test after 20 update loops, and the other two recovered. On EvenOdd, BASIL scored 20/20 at both endpoints on the symmetry-aware OOD test, versus 0/20 at both endpoints with predicate reuse. Bijectivity rules out mergers during search, but only one of the 120 bijections on Half satisfies all seven training sums. Reasoning supervision selects among them, and downstream learning is unconstrained.

For DeepStochLog on Half, 18/18 initially aligned runs retained the five digits at epoch 20, and one of the other two recovered. In the final failure two Stage I clusters already had the same majority digit. In the first epoch ID reasoning accuracy rose from 49.0% to 88.0% while digit accuracy fell from 63.7% to 45.2%. The learner reinforced a merger that fit every training sum. Final success was 19/20 on both ID and OOD tests (Appendix F.2.1).

On Kand-Logic, aligning the same pretrained models raised mean test reasoning accuracy from 60.2% to 99.9% before any downstream update. All 20 models retained the nine categories at epoch 40 under the naming fixed at initialization. All 17 entropy-only controls reaching at least 99% test accuracy used just three labels (Appendix F.6).


Q5 (Question 1 and Weakness 7). How does BASIL ground Kand-Logic, handle its symmetries and preserve color?

A5. Stage I uses contrastive pretraining to group the released object crops into nine clusters. Stage II uses the training outcomes to align them to the nine shape–color categories. Summing the 3×33\times3 probability table along its axes gives the official reasoner shape and color probabilities. Stage I preserves color with mild geometric transformations and padding, omitting grayscale conversion and color jitter.

Permuting shape names, permuting color names and exchanging the two attributes gives 2(3!)2=722(3!)^2=72 global bijections. All 72 preserve the OR answer on every configuration. The released examples also admit 72 non-injective maps that copy one attribute into both outputs. Each fails somewhere on the full rule. BASIL's unrelabeled test reasoning accuracy was 100.0%. Joint concept accuracy was 98.6% under one validation-selected symmetry and 15.3% under the original names (0/20 original-name successes). The global symmetries explain why the choice of names does not change the reasoning answer (Appendices E.3 and F.6).

AuthorsOpenReview ↗

Response to Reviewer aaWc (3/3)

Q6 (Weaknesses 4 and 5 and Question 2). Can BASIL handle BDD-OIA, relations learned from perception, or an unknown vocabulary size?

A6. BDD-OIA's 21 binary concepts can co-occur. BASIL currently aligns mutually exclusive categories. We therefore did not evaluate BDD-OIA. A joint vocabulary could require up to 2212^{21} categories. A multi-label output could represent the co-occurring concepts directly. Kand-Logic demonstrates composition of a small joint vocabulary under a supplied rule. Learning the relations themselves from raw scenes would also require grounding those relations (Appendix E.4).

When the knowledge base lists the elementary categories, it supplies kk. If kk is unknown, the current method cannot infer it. An extension could use reasoning outcomes to decide which groups in a finer partition should be merged.


Q7 (Weakness 8). Why does a temperature of 1.1 affect Large-K, and how should users choose the search parameters?

A7. At k=50k=50, τg=1.1\tau_g=1.1 is the high-temperature extreme of our sweep. The 10% increase from the default is a substantial push toward exploration. It cuts the maximum odds of a cluster first proposing one predicate over another from 451:1 to 259:1 (43%). In the single-mode limit those odds likewise fall from 316:1 to 187:1. The 15/20 benign Stage II outputs and 16/20 downstream successes were therefore obtained at a setting that sharply limits how strongly a cluster can favor its preferred predicate (Appendix F.5).

We recommend the shared defaults as a starting point. During development we chose ρ=0.1\rho=0.1, η=0.1\eta=0.1 and T=50T=50 together on Decipher and Large-K using semantic recovery, and retained τg=1\tau_g=1 and λ=0.7\lambda=0.7 with 50,000 candidates. We transferred these settings to Kand-Logic without retuning Stage II. In Figure 5, 54 of the 60 displayed task–parameter points gave 20/20 downstream successes, with default runs reused across sweeps. At a fixed partition and budget, held-out outcomes can reveal losses in reasoning accuracy, although equal accuracy can conceal different concepts.


Q8 (Weakness 9). How much ambiguity does binding remove, and how many shared maps remain on each task?

A8. On Addition-Half, strong binding reduces 576 feasible occurrence-level assignments to three shared maps. Bijectivity leaves one. Take one copy of each of the seven training pairs. With 14 independent occurrences a digit can change its name from one pair to the next. Tying repeated occurrences forces a shared name, and two of the three surviving maps merge distinct digits.

Table 8 counts shared deterministic maps preserving the stated training support under perfect binding. Bijective counts include identity. Large-K counts apply to the evaluated relations.

TaskBijectiveNon-injective
Addition10
Decipher10
Addition-Half12
Addition-EvenOdd234
Large-Kkk0
Kand-Logic7272

Q9 (Weakness 10). How does strong binding relate to cluster separability and similarity-based ABL?

A9. BASIL uses similarity to form groups and reasoning outcomes to name them before downstream training. Andolfi and Giunchiglia rely on separable concepts and use concept-labeled support examples to give prototypes their meanings. Huang et al.'s ABLSim uses similarity to select consistent abductions during training. BASIL pools reasoning constraints across each group to obtain a classifier that can initialize other learners. Section 1 illustrates shortcut collapse and cites the survey by Marconato et al. Appendix A discusses these connections and the role of concept-preserving augmentations.


We welcome further discussion of the evaluation and the scope of the theory.

Reviewer_WTdGOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces a new technique for addressing reasoning shortcuts in Abductive Learning. This technique combines an existing clustering approach with an alignment phase. Experiments are performed mostly on visual tasks with logic rules that encode addition-like tasks, and compare the approach with ablations.

Critical strengths

The results on the datasets are quite strong.

The paper has an extensive appendix with experiments, additional related work, etc.

Critical weaknesses

The experiments do not convincingly support the paper’s broad claims. This is the most critical weakness.

  • All visual experiments use similar logical theories (rules of addition). Large-K introduces a different theory, but is not a visual experiment. Consequently, the experiments provide limited evidence that BASIL works with different knowledge bases.
  • The paper lacks convincing comparisons against existing reasoning-shortcut mitigation methods. The authors only compare with existing NeSy systems like DeepStochLog and A3BL, but do not compare with reasoning-shortcut methods except for an ablation of their own approach.
  • Decipher shows a disconnect between the theory and experiments. Under the paper’s "perfect strong-binding" definition, Decipher has no (non-trivial) benign nor malignant shortcuts under perfect strong binding.

Important aspects of the method are insufficiently explained in the main paper, and the writing is sometimes imprecise.

  • In Equation 4, "U" is the random utility matrix used by the proposal sampler. It is defined by the equation and later in the paper implicitly called "utility" (I assume), but this should be stated before the equation.
  • The quantities "m" and "rho" in lines 235–237 are not introduced in the main text.
  • It is not explained how the authors "select one entry per row and column to maximize (...)" (Eq 6).
  • The authors cite Marconato et al. for L+PT, which is misleading. L+PT uses part of the authors’ own method and is a BASIL ablation, not work by Marconato et al.
  • The paper combined BASIL initialization with DeepStochLog (and other systems), but doesn't explain how. Especially since DeepStochLog is not an ABL method, it is unclear how to do this as it is not explained.

The practical contribution of the theory is limited and insufficiently explained. Theorem 2 isn't really motivated and Theorem's 3 usefulness is also unclear. If they are useful, the paper does not explain this in a way that makes it clear why.

Questions for authors

See the weaknesses above.

2 comments
AuthorsOpenReview ↗

Response to Reviewer WTdG (1/2)

We thank Reviewer WTdG for recognizing the strength of the experimental results and the breadth of the supporting experiments and related work in the appendix. The revision extends that evidence to a different knowledge base and a published shortcut-mitigation method.

Summary of the changes.

  • Non-arithmetic comparison (A1). On Kand-Logic, models starting from the same pretrained predictors recovered all nine categories in 20/20 runs with BASIL and 0/20 with concept entropy alone.
  • Theory and implementation (A2–A5). We distinguish when the objective favors the correct assignment from when the search finds it. We also clarify the sampler, matching solver, pretraining control and transfer to DeepStochLog.

New or rewritten main-text passages and the titles of new appendix subsections are marked in blue.


Q1 (diversity of rules and external baselines). Does BASIL help on a non-arithmetic visual task compared with an existing mitigation method?

A1. On standard RSBench Kand-Logic, BASIL-initialized models could use either attribute to answer. High-accuracy models trained with marginal concept entropy alone relied on one.

Setup. All four conditions shared the pretrained nine-category predictor, official reasoner, training order and 40-epoch schedule within each of 20 seeds. BASIL put the outputs in predicate order using training outcomes. We chose the entropy weight for each initialization by validation outcome macro-F1 on three separate pilot seeds. Reasoning accuracy uses the original outputs. For concept metrics, one symmetry selected with validation concept labels is fixed for testing. Marconato et al., NeurIPS 2023

Result.

InitializationConcept entropyTest reasoning accuracy (%)Joint concept accuracy (%)Runs recovering all nine categories
PretrainingOff98.137.60/20
PretrainingOn99.339.50/20
BASIL-initOff100.098.620/20
BASIL-initOn100.098.620/20

With entropy off, initialization alone changes recovery from 0/20 to 20/20. The rule accepts agreement of either shape patterns or color patterns. These coincide on the released splits, so copying one attribute into both outputs can satisfy the rule and give both marginals high entropy. All 17 entropy-only controls attaining at least 99% test reasoning accuracy used just three labels. On recombined positives their mean recognition rates were 94.8% for each model's stronger branch and 0.3% for its weaker branch. BASIL recognized 91.7% of shape-only and 95.5% of color-only positives. Table 19 includes the native six-output architecture separately.

AuthorsOpenReview ↗

Response to Reviewer WTdG (2/2)

Q2 (Decipher and Theorem 2). If only identity solves Decipher under perfect strong binding, what does the experiment test?

A2. The intended naming is uniquely optimal on Decipher even with the observed cluster errors, yet searches with the same budget differ sharply in whether they find it. A collapsed assignment can still score well because mapping every digit to zero satisfies single-digit sums.

Theorem 2 bounds the score change of every bijection by the length-weighted probability that an equation contains a cluster error. An ideal gap larger than twice this quantity preserves the optimum. The fixed-data bound certifies all 20 MNIST checkpoints. On CIFAR-10 the bound is inconclusive, but scoring all 10!10! bijections confirms that the intended assignment is uniquely optimal on all 20 checkpoints (Appendix D.1).

In Figure 8 the three searches share the checkpoint and a budget of 50,000 candidates within each CIFAR-10 run. All return bijections. Uniform sampling found the intended assignment in 0/20 runs, one-shot weighted matching in 10/20, and BASIL in 20/20 before downstream training. Using scores to guide later samples is what the adaptive search adds.


Q3 (practical use of Theorem 3). What does Theorem 3 tell us about an actual search?

A3. It gives a certificate for the matching returned by a completed search. Different rounds can favor different valid namings whose pieces combine into an invalid matching. Using the recorded elite matrices, we compare one fixed benign naming with the strongest unacceptable matching at each round. If the discounted sum of its margins is positive, the final decoder must return a benign naming. The certificate was positive on all 57 trajectories with a bijective Stage I reference (20 Decipher, 17 Addition-Half, 20 Large-K). The three remaining Half runs lacked such a reference (Section 4.3 and Appendix D.2).


Q4 (Equations 4 and 6). What are UU, mm and ρ\rho, and how is the final assignment computed?

A4. Equation 4 defines UU as the matrix of temperature-scaled log proposal weights plus Gumbel noise. Each of TT rounds scores mm candidates and keeps the top fraction ρ\rho, giving M=mTM=mT samples. Equation 6 applies a linear assignment solver to −Q-Q to obtain the maximum-weight matching (Section 4.2).


Q5 (baseline and transfer). Whose baseline is L+PTL{+}\mathrm{PT}, and how does BASIL initialize DeepStochLog?

A5. L+PTL{+}\mathrm{PT} is our pretraining-only control. Marconato et al. supply the entropy regularizer in A1. On Addition-Half, BASIL fits a classifier to the decoded cluster pseudo-labels for five epochs and transfers its encoder and predicate-ordered head. The paired control loads the same pretrained backbone with a fresh head. DeepStochLog then optimizes its own differentiable program likelihood. A³BL likewise keeps its native training rule (Appendix E.4).


We welcome further discussion of the paired comparison and the theory.

Reviewer_7KHzOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Abductive Learning may suffer from the problem of reasoning shortcuts. This paper proposes BASIL (Binding-Alignment for Semantic Initialization via Logic-consistency) to solve this problem. BASIL consists of two steps: first, the system clusters inputs, then, the system performs semantic alignment by maximizing logical consistency. Authors also reported their experiments to demonstrate the promising of BASIL.

Critical strengths

BASIL breaks the end-to-end learning by introducing the intermediate conceptual representation. This intermediate representation bridges the data inputs and logical reasoning and theoretically solves the problem of reasoning shortcuts.

Critical weaknesses

My main concerns are as follows: it is too strong to claim that concepts are one-to-one mapped to data clusters; and experiments are too simple to claim the proposed method can solve the problem of reasoning shortcuts in real situations.

It is true that an object concept may ground to one data cluster, but scenery concepts may ground to a configuration of data clusters. For example, desks and chairs may ground to single data clusters, but office rooms may ground to a configuration of desks, chairs, and other furniture. Theoretically, considering this one-to-many grounding will dramatically increase the computational complexity.

The experiments reported in the paper are insufficient to support the soundness of the proposed method.

Questions for authors

please see the weankess part. Why do you need one-to-one mapping (bijective cluster-to-concept mapping)?

1 comment
AuthorsOpenReview ↗

Response to Reviewer 7KHz

We thank Reviewer 7KHz for recognizing how BASIL's explicit intermediate representation connects the inputs to logical reasoning and provides a principled way to address reasoning shortcuts.

Summary of the changes.

  • Which concepts receive a cluster assignment (A1). Section 2 distinguishes the elementary categories from the scene a rule defines over them.
  • What bijectivity buys (A2–A3). Figure 9 shows the cost of a digit merger on unseen sums. Appendix F.6 tests composition under a non-arithmetic rule over nine objects.

New or rewritten main-text passages and the titles of new appendix subsections are marked in blue.


Q1 (office example and search complexity). Why require a one-to-one mapping if a scene concept can be grounded in a configuration of objects? What does this imply for search complexity?

A1. The bijection keeps elementary categories distinct when training outcomes permit a merger (A2). A scene concept can still depend on several objects. Given object-level inputs and their spatial relations, BASIL assigns predicates such as desk and chair to clusters, and a supplied rule recognizes an office from their configuration. With kk elementary categories, Stage II has k!k! possible namings, although a more complex rule can cost more to evaluate. Our head predicts one category per input unit. Discovering the objects and relations in a whole-room input, or learning which configurations define an office, requires additional inference beyond that naming search (Section 2 and Appendix E.4).


Q2 (need for bijectivity). Why require distinct predicates if a many-to-one assignment fits the training outcomes?

A2. On Addition-Half, BASIL achieved 84.0% reasoning accuracy on unseen digit pairs, compared with 10.1% when alignment allowed predicate reuse.

Setup. The two methods shared the Stage I checkpoint and downstream training protocol within each of 20 paired runs.

Result. The variant used four of five predicates on average. It scored higher on training consistency (65.9% versus 56.6%) and on familiar pairs after training (88.0% versus 85.0%). The unseen sums exposed the cost of merging digits (Figure 9 and Table 15). The assignment also worked with noisy groups. On Decipher, all 80 runs met the class-by-class recovery criterion across settings with mean cluster purities of 51.8–67.7% (Appendix F.4).


Q3 (experimental scope). What evidence goes beyond the original tasks?

A3. BASIL preserved the categories needed to reason jointly over nine objects on standard RSBench Kand-Logic. The benchmark supplies crops of familiar objects and tests how their shape or color patterns combine across figures. Each of the 20 pretrained partitions represented all nine shape–color categories. After 40 epochs, 0/20 pretraining-only models recovered all nine, while 20/20 BASIL-initialized models retained them under the naming fixed at initialization (Appendix F.6).


We welcome further discussion of the grounding assumption and its scope.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their careful reading and thoughtful feedback. Our paper introduces BASIL, a two-stage initialization for Abductive Learning (ABL). A learner can give the right answer on every training example while merging concepts it will need to distinguish on new inputs. BASIL addresses this shortcut before downstream training begins. It first learns which inputs belong together, then searches for a one-to-one assignment of groups to logical predicates that explains the observed reasoning outcomes across examples.

Our new experiment on RSBench Kand-Logic shows why those distinctions matter beyond arithmetic. An example is positive if its three figures share a shape pattern or a color pattern. On the released splits these conditions always coincide, so either attribute suffices. With the same pretraining and marginal concept-entropy regularization, 17 of 20 runs without alignment reached at least 99% test accuracy using just three of the nine shape–color categories. All 20 BASIL-initialized runs reached 100% and kept all nine categories distinct. When we recombined test objects so that only shape or only color satisfied the rule, BASIL recognized over 90% of positives of each type. For the 17 high-scoring controls, that rate averaged just 0.3% when the answer depended on the attribute each model tended to ignore (Section 5.1 and Appendix F.6).

Our theoretical analysis explains why even a learner that keeps the concepts distinct may have no way to recover their original names. On Addition-EvenOdd, shifting every digit by five modulo ten preserves every training sum, yet changes the correct answer on 50 of the 84 unseen digit pairs. The two namings give the learner exactly the same training evidence. We prove that if unseen pairs are equally likely, any learner relying on this evidence must incur an expected reasoning error of at least 25/84 under one of the two namings (Section 3 and Appendix B).

This distinction changes how we evaluate Addition-EvenOdd. For each of ABL, A³BL and DeepStochLog, all 20 BASIL-initialized runs kept the ten digits distinct, and ten of the 20 also recovered their original names. When we scored the predicted sums directly across all runs, ABL's accuracy on unseen pairs rose from 5.2% with pretraining alone to 53.8% with BASIL. The two external learners saw comparable gains (Tables 12 and 13).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.