acceptodds
Under review as a conference paper at ICLR 2027

NExRI: Neural Explicit Rule Induction for Abstract Visual Reasoning

Abstract

Abstract visual reasoning requires inducing a relation from a few observed instances and re-applying it to novel ones. Existing models encode this relation implicitly in network weights, leaving it unobservable and unverifiable. We introduce **NExRI** (**N**eural **Ex**plicit **R**ule **I**nduction), a framework for end-to-end explicit rule induction on neural features, realized by two coupled modules. *IVaR* constructs contextual states that expose the latent relation by predicting the third panel from the first two and iteratively refining the prediction-observation discrepancy through verify-and-refine updates. *RaXO* turns these states into explicit rule objects that are validated on unseen evidence and executed on candidate completions. Instantiated as polynomial equations, the induced rules keep the fit tractable and the whole framework differentiable end-to-end. Experiments demonstrate state-of-the-art performance across multiple abstract visual reasoning benchmarks.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Abstract Visual Reasoning, Out-of-Distribution Generalization, Representation Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_yqXPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes NExRI, a framework for abstract visual reasoning that constructs and executes explicit polynomial constraints on learned representations. Its IVaR module uses predictive discrepancies and recurrent verification and refinement updates to construct contextual row states. RaXO then generates polynomial hypotheses from complete context rows, evaluates their agreement on the other row, and jointly fits a shared constraint before scoring candidate completions. Training differentiates through this fitting procedure. The paper reports results on RAVEN-family benchmarks, eight PGM regimes, CLEVR-Matrices, and two Bongard benchmarks. Ablations investigate the major modules, hypothesis weighting, and readouts trained on frozen representations, while additional diagnostics examine coefficient transfer and fitting behavior.

Critical strengths

The induced constraints are part of the prediction mechanism and can be independently re-executed. RaXO derives its functions, references, and support weights from complete context rows and keeps them fixed when evaluating candidates. This provides a concrete approah for inspecting and intervening on the computations used to make predictions. The regularized fitting formulation also permits joint training with the visual representation, making this a useful architectural contribution.

The improvements can extend to meaningful generalization settings with useful component controls. On the four A-I-RAVEN splits, NExRI improves over DSRF-L by 2.3–6.5 percentage points, increasing the average from 71.5% to 75.5%. It also improves the average across eight PGM regimes from PredRNet’s 60.4% to 65.4%. The frozen-representation comparison reports 75.1% for RaXO versus 69.6% for the neural scorer, which provides evidence that the gains may involve the reasoning readout as well as representation learning.

Critical weaknesses
  1. Table 3 reports 97.3% on Bongard-LOGO and 96.8% on Bongard-OpenWorld, which substantially exceeds the listed baselines. However, Appendix D.1 gives only a high-level description of the adaptation. It does not define how positive and negative support examples and an individual query are mapped to IVaR’s three-position input, how support labels enter polynomial induction and fitting, or how execution costs produce a binary prediction.

  2. Table 8 explicitly describes only the frozen Type diagnostic configuration, leaving the dataset-specific encoder, pretraining, training budget, and model-selection procedures for these main results unclear.

  3. The central explicit-rule contribution needs more precise positioning and validation. The broad novelty claims should account for PrAE (jointly trains perception and rule-based reasoning) or ALANS (performs instance-specific operator fitting and execution with joint training).

  4. Figure 2 names the learned views Number/Position/Type/Size/Color, and Table 7 transfers coefficients between designated attribute views, but the described projections and variance/covariance regularization do not specify how those semantic assignments are obtained.

  5. Appendix D.1 mentions component and proxy labels, which require further explaination. The executable polynomial interface is well defined, but the evidence for transferring particular attribute rules depends on this missing correspondence.

Questions for authors
  1. How are the learned views and spatial components associated with the named attributes/components used in Figure 2 and Table 7?

  2. Are the component and proxy labels mentioned in Appendix D.1 used during training?

  3. What distinguishes NExRI from PrAE and ALANS’s differentiable operator fitting and execution?

  4. What are the inputs, architecture, capacity, and training procedure of the Neural Scorer in Table 6?

5 comments

yqXP-W1

Table 3 reports 97.3% on Bongard-LOGO and 96.8% on Bongard-OpenWorld, which substantially exceeds the listed baselines. However, Appendix D.1 gives only a high-level description of the adaptation. It does not define how positive and negative support examples and an individual query are mapped to IVaR's three-position input, how support labels enter polynomial induction and fitting, or how execution costs produce a binary prediction.

We thank the reviewer. Appendix D.1 describes the adaptation at the level of components and leaves the interface to IVaR implicit; we agree that this is insufficient to reconstruct the method. The revised appendix will state the three mappings explicitly, and we give them here. They are also the mappings used for the results in Table 3, so the numbers are unchanged.

yqXP-W1.1A

How are the supports and the query mapped to IVaR's three-position input?

IVaR consumes two context panels and one target panel; the adaptation changes what the panels contain, not the module. A shared attention-pooling module with two learned queries pools the six positive and the six negative supports separately, producing two anchors per class. A class's two anchors occupy the context positions, and the panel being related occupies the third. During fitting, each support row forms its own three-position input [ac1,ac2,ϕ(xic)][a_c^{1},a_c^{2},\phi(x_i^{c})], so a class yields six such inputs; the six are neither concatenated nor fed as six panels. During execution, the query is encoded once and paired with each class's anchors, giving [ac1,ac2,ϕ(q)][a_c^{1},a_c^{2},\phi(q)] for c∈+,−c\in\\{+,-\\}; both class-conditioned inputs are processed in one batch, so the two-class decision needs no second encoder pass. Support fitting completes before any query enters execution, so query information never reaches the programs.

yqXP-W1.2A

How do support labels enter polynomial induction and fitting?

Support labels enter induction as a partition of the fitting set, not as a supervised term in the objective. The six positive rows and the six negative rows are fitted as two independent six-row problems, so each class yields its own program and no support label appears in the loss that fits it; a positive row is related to positive context and a negative row to negative context, and the query enters neither proposals nor references. Joint Fit minimizes the mean squared residual over each class's own six rows, so the coefficients and references of the two programs are estimated from disjoint support sets. Discrimination between them is deferred to execution rather than expressed as a margin during fitting.

yqXP-W1.3A

How do execution costs produce a binary prediction?

Execution is deterministic: anchors, coefficients, references and slot weights are fixed before the query is scored. A query yields 60 execution costs per class, from 15 read/view slots and four hypotheses, each averaged over the three degrees; each cost dd enters pooling as −αd-\alpha d, with a shared positive scale α=1+softplus(a)\alpha=1+\mathrm{softplus}(a) trained jointly with the query cross-entropy. For each class, we take a log-mean-exp over the four hypotheses, then an average over the 15 slots with normalized support-derived sigmoid weights. The resulting class evidence enters the two-class softmax as its logit, and the prediction is the argmax. No threshold or temperature is tuned and no post-hoc calibration is applied: the boundary is fixed once the programs and weights are, before any query is executed, so α\alpha is the only calibration parameter and it is trained rather than tuned on the benchmark.

yqXP-W2

Table 8 explicitly describes only the frozen Type diagnostic configuration, leaving the dataset-specific encoder, pretraining, training budget, and model-selection procedures for these main results unclear.

We thank the reviewer and agree. Table 8 reports frozen Type diagnostics; the revised version will state this in its caption and add the configuration summary below as Table R2.

yqXP-W2.1A

Encoder, input and optimization for Bongard-LOGO.

A ResNet-12 trained from scratch, with 128-dimensional outputs and no pretraining, on 512×512512\times512 grayscale inputs. SGD with learning rate 10−310^{-3}, momentum 0.9, weight decay 5×10−45\times10^{-4}; batch size eight; 100 epochs. The total parameter count is 2.09M, against 8.10M for the released B-SC and ProtoNet ResNet-12, because the channel widths differ.

yqXP-W2.2A

Encoder, input and optimization for Bongard-OpenWorld.

A LAION-2B ConvNeXt-base, trainable rather than frozen, with 1,024-dimensional outputs on 224×224224\times224 RGB inputs. AdamW with learning rate 10−410^{-4} and weight decay 5×10−45\times10^{-4}; batch size ten; 20 epochs including five warmup epochs followed by cosine decay. The parameter count is 88.1M, comparable to MetaOptNet (88.2M) and smaller than the caption-enabled ProtoNet (320.1M) and SNAIL (347.9M).

yqXP-W2.3A

Model selection.

Selection uses validation query accuracy only, computed with the same support/query organization as training. On LOGO the validation split is traversed in full with fixed queries, so every epoch is scored on the same episode set and the comparison across epochs is paired; on OpenWorld each of the three runs selects its own best validation checkpoint. Test data enter only the reported numbers. No hyperparameter was searched on either benchmark: both schedules and both optimizer settings are the official recipes [5, 6], applied without benchmark-specific tuning.

Table R2. Configuration of the Bongard experiments in Table 3. Models are trained per benchmark; the frozen Type checkpoint behind Table 8 is not involved. Parameter counts are for the full model.

SettingBongard-LOGOBongard-OpenWorld
EncoderResNet-12, 128-dConvNeXt-base, 1,024-d
PretrainingnoneLAION-2B, fine-tuned
Input512×512512\times512, grayscale224×224224\times224, RGB
OptimizerSGD, lr 10−310^{-3}, mom. 0.9, wd 5×10−45\times10^{-4}AdamW, lr 10−410^{-4}, wd 5×10−45\times10^{-4}
Batch size810
Schedule100 epochs20 epochs, 5 warmup + cosine
Parameters2.09M88.1M
Objectivelabel-smoothed query CE +,0.1+\\,0.1 VICReg-style regularization, uniform hypothesis priorslabel-smoothed query CE +,0.1+\\,0.1 VICReg-style regularization, uniform hypothesis priors
Augmentation, caption supervisionnonenone
Checkpoint selectionvalidation query accuracy, full split, fixed queriesvalidation query accuracy, per run

yqXP-W3

The central explicit-rule contribution needs more precise positioning and validation. The broad novelty claims should account for PrAE (jointly trains perception and rule-based reasoning) or ALANS (performs instance-specific operator fitting and execution with joint training).

Our contributions are threefold. 1) We introduce NExRI, an end-to-end framework that unifies explicit rule modeling with neural feature learning: relations are verifiable polynomial invariants on learned states, trained jointly by differentiating through the fitting step. 2) IVaR, a recurrent reasoning block, produces contextual states that expose the latent relation through coupled predict-and-verify and verify-and-refine loops. 3) RaXO turns these states into explicit rule objects: each complete row proposes a function and a reference, validated on the other row before a common constraint is fitted and executed on candidates.

The two closest explicit-rule methods differ at the point where a candidate is produced and how it is selected. PrAE [8] abduces predefined logical rules over learned attributes, so a relation is chosen from a fixed vocabulary; NExRI instead regresses free polynomial coefficients and references on contextual row states, and the constraint defining the rule is the object the optimiser differentiates. ALANS [9] fits operators within a given space, independently per instance; RaXO validates a proposal on the row that did not generate it and retains its support as a soft weight over all four hypotheses under one fitted criterion. IVaR keeps the states comparable across rows, which is what makes that cross-row validation well defined.

yqXP-W3. Benchmark comparison

Following the reviewer's request, Table R3 reports results for PrAE and ALANS under their original RAVEN and I-RAVEN evaluation protocols [8, 9] and the benchmark authors' protocol for Mesh and the four A-I-RAVEN OOD splits [4]. Both methods score substantially lower on the held-out splits under the benchmark authors' protocol. ECo-NSR is cited as related work but does not supply a full-system row, since its paper states that end-to-end inference is not yet available [3].

Table R3. Accuracy (%) on RAVEN-family benchmarks. See the table note for evaluation protocols.

MethodRAVENRAVEN-FAIRI-RAVENI-RAVEN-MeshOOD ColorOOD PositionOOD SizeOOD TypeAvg.
PrAE (2021)65.0388.3077.0233.20 ± 0.4047.90 ± 0.9068.20 ± 3.3041.30 ± 1.8037.00 ± 1.7057.24
ALANS (2022)74.4090.3178.5015.90 ± 2.6015.20 ± 1.4016.00 ± 1.0023.30 ± 6.5019.00 ± 3.4041.58
NExRI (Ours)99.699.799.795.1 ± 0.873.1 ± 0.982.1 ± 0.784.5 ± 0.962.1 ± 1.187.0

RAVEN and I-RAVEN entries are as reported in [8, 9], with PrAE's RAVEN-FAIR result from [10]. Mesh and A-I-RAVEN baselines are mean and SD over three seeds under the shared recipe of [4]. Avg. is the mean over the eight settings.

yqXP-W4

Figure 2 names the learned views Number/Position/Type/Size/Color, and Table 7 transfers coefficients between designated attribute views, but the described projections and variance/covariance regularization do not specify how those semantic assignments are obtained.

We do not claim that the model obtains a semantic assignment, nor that its latent space is decodable into attributes. The attribute names in Figure 2 are the generator's definitions (Number, Position, Type, Size, Color), human-specified and fixed before any model is trained; no attribute label enters the forward pass, the loss, or any supervision signal, and VICReg-style variance and cross-view covariance regularization treats the five views symmetrically.

The test is not whether the model recovers human-interpretable attributes or rules, but which generator attribute drives each projection. We freeze the checkpoint and, for each attribute, apply a single-attribute edit while holding the remaining attributes and the rendering seed fixed, recording the L2L_2 change of each validation-standardized projection feature on validation data. Table R1 reports the resulting correspondence. Rows sum to one across the five views. On validation data, each attribute's edit identifies the view with the largest response. These five maxima identify distinct views, which we name and order by the corresponding attributes. Table 7 transfers coefficients between these labelled slots. The labelling step fixes only the slot names and does not enter training.

Table R1. View–attribute correspondence.

Changed attributeNumberPositionTypeSizeColor
Number0.770.100.020.070.04
Position0.080.810.020.050.04
Type0.060.050.750.090.05
Size0.040.050.060.830.02
Color0.030.030.010.040.89

Projection-feature responses to single-attribute image edits, standardized on validation data. Rows are normalized across views.

yqXP-W5

Appendix D.1 mentions component and proxy labels, which require further explaination. The executable polynomial interface is well defined, but the evidence for transferring particular attribute rules depends on this missing correspondence.

Appendix D.1 defines the component/proxy-label correspondence under augmentation: a component label records which source component each part of an augmented or composed sample is built from, and the source-component proxy label records the corresponding component in the source sample. Neither enters any training loss: both are constructed offline and are never read by the training loop, and no loss term, auxiliary head, or data-loader target consumes them. Their only role is diagnostic and bookkeeping, and they are not used for checkpoint selection or hyperparameter choice. Training optimizes answer cross-entropy plus label-free VICReg-style regularization, and no annotation beyond the answer class label reaches any loss.

The correspondence that the transfer in Table 7 depends on is the view–attribute correspondence, which is established independently of these labels and is measured in our response to yqXP-W4 (Table R1). Table 7's coefficient interventions act on fixed view/read indices defined by that frozen assignment; generator rule labels define the same-rule and different-rule groups for reporting, independently of the proxy labels.

Reviewer_snvDOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes NExRI, a framework for explicit rule induction in abstract visual reasoning. IVaR constructs contextual features through iterative verification and refinement, while RaXO induces polynomial rules through cross-row verification and joint fitting, then uses them to evaluate candidates. The method is evaluated on several reasoning benchmarks.

Critical strengths
  1. NExRI combines neural feature learning with explicit rule induction. The induced polynomial rules can be verified and executed on new inputs while remaining end-to-end trainable.
  2. The method achieves strong results across multiple benchmarks, including OOD settings. Ablation studies also support the contributions of IVaR and RaXO.
Critical weaknesses
  1. The induced rules operate in a learned latent space. Table 7 provides initial evidence of within-rule agreement and coefficient transfer, but their correspondence to human-interpretable semantic rules remains only indirectly established.

  2. The introduction briefly discusses the limitations of large VLMs, but given the progress of frontier VLMs with reasoning harnesses, NExRI’s advantages and limitations relative to such systems deserves further discussion. Comparisons with such systems are missing.

Questions for authors
  1. Can the authors further examine whether the induced polynomial rules correspond to the underlying semantic rules and remain consistent across different instances?

  2. What challenges does NExRI address that remain difficult for general-purpose VLMs with specified reasoning harnesses? How does it compare in terms of generalization or reliability?

3 comments

Response to Reviewer snvD

snvD-W1

The induced rules operate in a learned latent space. Table 7 provides initial evidence of within-rule agreement and coefficient transfer, but their correspondence to human-interpretable semantic rules remains only indirectly established.

NExRI induces polynomials over learned feature views. The benchmark generator defines the semantic attributes and rule families used to evaluate the induced programs. On the 64 Center–Type puzzles in Table 7, row-support distributions agree more closely across puzzles sharing a rule than across shuffled pairs (Jensen–Shannon divergence ×100 of 8.8 versus 19.3). In the coefficient-transfer control on those puzzles, accuracies are 71.9% for native coefficients, 70.3% for same-rule donors and 62.5% for different-rule donors. Q1 measures semantic agreement by executing the induced programs on rule-valid completions and single-attribute violations.

snvD-W2

The introduction briefly discusses the limitations of large VLMs, but given the progress of frontier VLMs with reasoning harnesses, NExRI's advantages and limitations relative to such systems deserves further discussion. Comparisons with such systems are missing.

Table R1 reports published image-input evaluations of visual rule induction. The RAVEN rows use a 140-puzzle subset and the StemBind rows use a separate collection of 2,298 full-image items. Each figure uses its source study's items and harness. A controlled comparison with NExRI would require the same test puzzles and evaluation protocol.

Table R1. Published VLM accuracy on visual rule induction.

ModelTest collectionHarnessAccuracy (%)
GPT-5RAVEN, 140 puzzlesNative reasoning + CoT27.86
Gemini 3 ProRAVEN, 140 puzzlesNative reasoning + CoT40.00
Gemini 3.1 ProStemBind, 2,298 full-image itemsDirect mode35.90
Claude Opus 4.7StemBind, 2,298 full-image itemsDirect mode30.70

Sources for the RAVEN and StemBind rows, respectively [1, 3].

NExRI jointly learns visual representations and fits an executable polynomial program to each puzzle's context. Its coefficients, references and support weights stay fixed across all candidate completions, so we can replay their execution and test how coefficient substitutions affect the answer. This design requires task-specific training and confines the induced relations to the chosen polynomial family. A general-purpose VLM offers broader visual and language coverage, and a tool-enabled harness can execute rules that the VLM generates.

snvD-Q1

Can the authors further examine whether the induced polynomial rules correspond to the underlying semantic rules and remain consistent across different instances?

Correspondence. For each eligible puzzle, the program fitted on the two complete context rows scores one generator-valid completion and all eligible violations obtained by changing only the held-out attribute. AUROC measures the fraction of valid–violating pairs in which the valid completion receives stronger pre-softmax execution evidence, with ties counted as one half. We average puzzle-level AUROC within each generating rule family, then weight the families equally. Macro-AUROC is 0.870 against a chance level of 0.500 (Table R2).

Table R2. Semantic-rule agreement.

Rule familyContextsAUROC
Progression8,0000.880
Arithmetic6,0000.830
Distribute Three10,0000.900
Macro-average24,0000.870

The three rule families are the nonconstant test rules for the held-out attribute, which follows Constant during training and validation (Appendix D.2.2).

Consistency. On eligible targets across the four A-I-RAVEN splits, same-rule donors reach 72.14% versus 69.05% for different-rule donors (Table R3). The paired difference is 3.09 percentage points (95% interval [2.66, 3.52]), with the same ordering on each split. This shows that the transferred coefficients retain rule-specific predictive information across instances.

Table R3. Cross-instance function transfer.

Function sourceColorPositionSizeTypeAvg.
Native70.91 ± 0.8780.13 ± 0.8681.83 ± 1.0960.11 ± 1.2173.25
Same-rule donor69.81 ± 1.0179.19 ± 0.9680.58 ± 1.1958.97 ± 1.2872.14
Different-rule donor66.91 ± 1.1276.37 ± 1.1377.17 ± 1.3355.73 ± 1.3869.05

Accuracy (%) with SD over three runs. Avg. weights the four splits equally. Donor coefficients replace the target attribute's coefficients across all three spatial reads. The three conditions share the same eligible targets, target states and uniform hypothesis weights. Each condition recomputes references on the target's context rows. Both donor groups are matched to the target on layout, component cardinalities and non-target rule signatures.

snvD-Q2

What challenges does NExRI address that remain difficult for general-purpose VLMs with specified reasoning harnesses? How does it compare in terms of generalization or reliability?

Challenges. A-I-RAVEN tests attribute–rule pairings held out during training. Solving these puzzles requires retaining visual distinctions such as size and position, then evaluating competing rules that can explain the same context. IVaR builds contextual states from images through recurrent prediction and verification. RaXO generates four hypotheses per read/view slot from each complete row, checks them on the other row, and jointly fits the paired proposals across both rows. A VLM harness can sample hypotheses or call an executor. NExRI trains its representations through differentiable fitting and scores every candidate with the same fitted program and support weights.

Generalization and reliability. NExRI reaches 75.5% across the four A-I-RAVEN held-out attributes and 65.4% across eight PGM regimes, compared with the strongest baseline means of 71.5% (DSRF-L) and 60.4% (PredRNet), respectively (Table R4). Cross Check supplies support weights using a row outside the proposal's induction context. Joint Fit then uses the two complete rows, and the fitted program scores each candidate with the same criterion and exposes its execution costs by component and view. The transfer control in Table R3 tests whether that criterion retains rule-specific information on new instances. W2 gives the corresponding discussion of VLMs and their published protocols.

Table R4. Generalization accuracy (%).

MethodA-I-RAVENPGM
PredRNet55.260.4
DSRF-L71.554.9
NExRI75.565.4

A-I-RAVEN averages the Color, Position, Size and Type columns of manuscript Table 1. PGM averages eight regimes as in manuscript Table 2. PredRNet's A-I-RAVEN entries come from the benchmark authors' three-seed single-task evaluation [5].

References

[1] Faiz Ghifari Haznitrama, Faeyza Rishad Ardi, and Alice Oh. A Neuropsychologically Grounded Evaluation of LLM Cognitive Abilities. arXiv:2603.02540, 2026.

[3] Xixiang He, Baiqi Wu, Xingming Li, Ao Cheng, Qiyao Sun, Xuanyu Ji, and Qingyong Hu. StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoning. arXiv:2606.00148, 2026.

[5] Mikołaj Małkiński and Jacek Mańdziuk. A-I-RAVEN and I-RAVEN-Mesh: Two New Benchmarks for Abstract Visual Reasoning. arXiv:2406.11061, 2024.

Thank you for the additional experiments and clarifications. I maintain my original rating.

Reviewer_GQ2dOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces NExRI for abstract visual reasoning. Its central idea is to induce puzzle-specific executable polynomial constraints over learned neural representations and select answers according to how well their completed rows satisfy these constraints. The framework contains two main modules. IVaR constructs relational states by predicting the third panel, computing prediction residuals, and iteratively updating verification and refinement states. RaXO proposes polynomial rules from the two complete context rows, cross-checks each proposal on the other row, and jointly fits a common constraint. Rules and aggregation weights remain fixed during candidate evaluation. Training differentiates through the fitting procedure. The paper reports strong results on the RAVEN family, PGM, Bongard benchmarks, and CLEVR-Matrices, together with module ablations, frozen-representation comparisons, support-weight analyses, and coefficient-transfer interventions.

Critical strengths
  1. The method provides a concrete, intervenable interface between context-based induction and candidate execution. RaXO represents a rule through polynomial functions and reference values, with context fitting completed before candidate scoring. Equations (8)–(12) distinguish proposal generation, cross-checking, joint fitting, and execution. The query row and candidate answers are excluded from rule fitting. This provides a more concrete interface for mechanism analysis than a compatibility score alone. Coefficients, references, and support weights can be manipulated separately while holding other components fixed, which is a meaningful methodological contribution.
  2. The reported performance is competitive, with improvements beyond nearly saturated standard benchmarks. In Table 1, NExRI outperforms the listed baselines on all four A-I-RAVEN attribute-generalization splits, achieving approximately 75.5% average accuracy compared with approximately 71.5% for DSRF-L. Table 2 reports an average of 65.4% across eight PGM regimes, exceeding the listed methods. The module ablations and frozen-representation comparisons in Table 6 also support the usefulness of RaXO. These results make the approach worthy of serious consideration.
Critical weaknesses
  1. The most substantial Bongard results lack sufficiently complete implementation and evaluation details. Table 3 reports 97.3% average accuracy on Bongard-LOGO and a four-category average of 96.8% on Bongard-OpenWorld, representing large improvements over the listed baselines. However, Appendix D.1 provides only a high-level adaptation description and explicitly omits the RPM-specific Cross Check. Important details remain unclear: How are six positive examples, six negative examples, and queries organized for IVaR, whose original formulation accepts three panels? How does the support set generate polynomial proposals and references? How do negative examples enter the fitting or discrimination objective? How is a query execution score converted into a binary prediction, and how are thresholds or calibration parameters determined? What encoder, pretraining, input configuration, training budget, and checkpoint-selection rule are used for each Bongard benchmark? Table 8 explicitly describes the frozen Type diagnostics and does not supply a complete configuration for these main experiments. Appendix D.2 also states that baselines retain their published pretraining, supervision, and optimization budgets. Consequently, the extent to which the large gains arise from the reasoning mechanism remains difficult to assess. These omissions do not establish that the results are incorrect. They nevertheless prevent adequate evaluation and reproduction of the paper’s strongest cross-task results and materially affect my recommendation.
  2. The ablations support the framework’s effectiveness, but evidence for the central mechanism and its distinctive contribution remains limited. I accept that the method implements explicit, re-executable latent constraints in an operational sense. Nevertheless, execution operates on learned representations that incorporate the candidate completion. Fixing polynomial coefficients alone does not sufficiently distinguish the benefit of transferable rule induction from that of learned relational representations combined with a structured scoring function. The paper does provide relevant evidence. RaXO outperforms the Neural Scorer on frozen representations in Table 6, and Appendix E.1 includes support-weight and coefficient-transfer interventions. However, these findings have important limits: The Neural Scorer’s architecture, capacity, and training settings are insufficiently specified. Interpretation of the frozen comparison also depends on how its IVaR representations were trained. The fixed-checkpoint support-weight comparison uses 448 puzzles. The 95% interval for the accuracy difference between pre-fit and uniform weighting is [−0.89, 2.90] percentage points. Joint fitting produces only three additional correct answers relative to proposal averaging, with an interval of [−1.12, 2.68] percentage points. Lower residuals on fitted context rows do not independently establish better query generalization. Coefficient transfer covers 64 Center puzzles and replaces the Type view of only the first spatial component, leaving other components and views unchanged. This supports local rule-related sensitivity but does not establish broad rule transfer. Furthermore, the paper acknowledges that its polynomial instantiation follows prior work, while related explicit rule-induction methods are absent from the main performance tables. More precisely positioning the contribution around the present combination of induction, verification, and differentiable fitting, supported by targeted comparisons, would strengthen the novelty assessment.
Questions for authors
  1. Implementation clarification concerning W1. Please provide the complete Bongard forward procedure or pseudocode, including example grouping, the use of positive and negative support examples, fitting objectives, query classification, and dataset-specific training and pretraining configurations. Please also report overall OpenWorld test accuracy aggregated over actual test examples. As the appendix acknowledges, the four-category average in Table 3 combines overlapping partitions and differs from overall accuracy.
  2. Experimental provenance and comparison conditions. How many random seeds underlie each main result, and what constitutes a replicate for the reported standard deviations? What training duration and checkpoint-selection rule are used? Do the component labels and source-component proxy labels mentioned in Appendix D.1 enter any training loss, or are they used only for diagnostics or bookkeeping? These answers affect the assessment of supervision conditions and reproducibility.
  3. Mechanistic evidence concerning W2. Please specify the Neural Scorer and justify its capacity matching. Can the fixed-model support-weight, joint-fitting, and coefficient-transfer interventions be evaluated across additional checkpoints or training seeds? The consequential question is whether these operations consistently improve predictions on queries excluded from fitting, rather than merely improving context agreement.
1 comment

GQ2d-W1

We thank the reviewer for this careful reading. We agree that the current manuscript does not describe the Bongard adaptation in sufficient detail. We have conducted the additional experiments requested in the review and report them below; the corresponding text and Table R1 will be added to the revised version.

GQ2d-W1.1A

How are six positive examples, six negative examples, and queries organized for IVaR, whose original formulation accepts three panels?

Each Bongard problem contains two queries and two classes of six supports. An attention-pooling module with two learned queries and shared parameters is applied separately to the six positive and the six negative supports, producing two anchors per class. The two class-conditioned inputs, each formed from a query and its class's two anchors, are processed together in a batch.

IVaR is still called with three positions, exactly as in its original formulation; only what each position holds has changed.

GQ2d-W1.2A

How does the support set generate polynomial proposals and references?

Proposals and references are both generated from the support rows; the query enters neither.

Proposals. Each support row yields four hypotheses. Its 3×83\times8 IVaR states are flattened to 24 dimensions and mapped by a shared 24→96→11224\to96\to112 GELU MLP, so the four slots share one trunk and differ only in their output units. The 28 outputs of a slot are the free coefficients of three polynomials of degree one to three over the attribute coordinates: for degree dd we take all nonconstant monomials of total degree at most dd and fix the coefficient of cdc^d to 1, leaving 2+8+18=282+8+18=28 across the three degrees. All outputs pass through tanh⁡\tanh under degree-specific scales.

References. A reference is not an external target and is not read from the query. A proposal's initial reference is its own output on the row that generated it, so that row's residual is zero by construction. Joint Fit then optimizes one shared polynomial and reference per class, hypothesis and degree against the six support rows, so the fitted pair is determined by the support set as a whole. The four hypotheses are weighted uniformly, while the coefficient and reference priors are centred on the proposal means. Positive and negative supports define separate programs.

GQ2d-W1.3A

How do negative examples enter the fitting or discrimination objective?

Fitting. The six positive support rows and the six negative support rows are fitted independently. Each class obtains its own polynomial and reference per hypothesis and degree by minimizing the mean squared residual over its own six rows, so the coefficients and references of the two programs are estimated from disjoint support sets: a negative row shapes no positive-class parameter and vice versa.

Discrimination. Each query is executed against both fitted programs. Both classes apply the same positive scale to execution cost, before pooling, and for each class we then negate the scaled costs, take a log-mean-exp over its four hypotheses, and average over the read/view slots with that class's support-derived weights to obtain that class's logit. The sign change is what turns a cost into evidence, and it is applied to the positive class as well as the negative one, so the two logits are produced by the same construction. A two-class softmax over them gives the query probabilities, and label-smoothed cross-entropy against the query label is the discrimination objective.

GQ2d-W1.4A

How is a query execution score converted into a binary prediction, and how are thresholds or calibration parameters determined?

Each query is executed against both fitted programs, giving an execution MSE per class, read/view slot and hypothesis. The scale α=1+softplus(a)\alpha=1+\mathrm{softplus}(a) is shared by both classes and learned through query cross-entropy. Each execution cost dd enters pooling as −αd-\alpha d. Pooling is a log-mean-exp over the four hypotheses followed by that class's support-weighted average over slots, which yields the class evidence scs_c. No threshold or temperature is tuned: scs_c enters the two-class softmax as the logit at unit temperature, so α\alpha is the only calibration parameter, and the prediction is the higher-probability class, equivalently the sign of s+−s−s_+-s_-. The boundary is therefore fixed once the programs and weights are, before any query is executed, and no post-hoc calibration is applied.

Comments on the paper

1 comment

GQ2d-W2(c). Statistical strength of the weight and fitting controls.

The fixed-checkpoint support-weight comparison uses 448 puzzles. The 95% interval for the accuracy difference between pre-fit and uniform weighting is [−0.89, 2.90] percentage points. Joint fitting produces only three additional correct answers relative to proposal averaging, with an interval of [−1.12, 2.68] percentage points. Lower residuals on fitted context rows do not independently establish better query generalization.

We have rerun both controls at full scale: every split now contains all 14,000 test puzzles, with mean and SD over three matched checkpoint/readout runs, and the weight controls share fitted programs and candidate costs at λ=1\lambda=1.

Support weighting. Table R4 reports the full-scale weight controls over all 14,000 puzzles per split and three matched runs. Pre-fit support gives 75.10 against 73.71 for uniform support, a difference of 1.39 points, or roughly 780 of the 56,000 test puzzles. The difference is positive on every split, at +1.16+1.16, +1.14+1.14, +1.82+1.82 and +1.36+1.36 on Color, Position, Size and Type, whereas the between-run SD across the same splits spans 0.70 to 1.09 points; every per-split difference thus exceeds the SD of both conditions on its split. Post-fit support reaches 74.57, trailing pre-fit on every split by 0.39 to 0.64 points, so the gain tracks when the weights are applied rather than whether they are fitted. Varying λ\lambda away from 1 does not improve on it, with 74.08 at λ=0.5\lambda=0.5 and 74.74 at λ=2\lambda=2, so the reported setting is not a tuned optimum.

Table R4. Full OOD controls.

VariantColorPositionSizeTypeAvg.
RaXO + pre-fit support72.70 ± 0.7081.70 ± 0.8084.10 ± 1.0061.80 ± 0.9075.10
RaXO + uniform support71.54 ± 0.9480.56 ± 0.8282.28 ± 1.0960.44 ± 0.9873.71
RaXO + post-fit support72.17 ± 0.7781.31 ± 0.9483.46 ± 1.0261.33 ± 0.9774.57
Capacity-matched Neural Scorer68.58 ± 0.9777.94 ± 0.6778.88 ± 0.9254.56 ± 1.1469.99
RaXO + λ=0.5\lambda=0.571.51 ± 0.9380.96 ± 0.8383.16 ± 0.9660.67 ± 1.1274.08
RaXO + λ=2\lambda=272.24 ± 0.8181.67 ± 0.7683.74 ± 1.0361.29 ± 1.0174.74

Query accuracy (%), mean and SD over three matched runs. Avg. equally weights the four splits and uses unrounded per-split means. The pre-fit row is the frozen IVaR + RaXO row of Table 6, and the capacity-matched row is repeated from Table R3; both are shown for reference, not re-fit. Weight controls use λ=1\lambda=1 and share fitted programs and candidate costs; fitting controls fix proposals and pre-fit support.

GQ2d-W2(c). Joint fitting

Joint fitting. To isolate context fitting, we compare Joint Fit at λ=1\lambda=1 with proposal averaging at λ=0\lambda=0, using the same frozen representations, pre-fit support weights, and aggregation. Joint Fit estimates shared coefficients and references from the two complete context rows. Proposal averaging takes the arithmetic mean of their coefficient and reference proposals. Both programs are fixed before scoring any candidate. In Table R5, Joint Fit improves query accuracy by 0.52, 0.55, 0.65, and 0.62 percentage points on Color, Position, Size, and Type, respectively, for a mean gain of 0.59 points. The full Type split's 0.62-point gain is close to the 0.67-point gain on the 448-puzzle Type diagnostic.

Table R5. Joint-fitting comparison.

ReadoutColorPositionSizeTypeAvg.
Joint Fit72.70 ± 0.7081.70 ± 0.8084.10 ± 1.0061.80 ± 0.9075.10
Proposal averaging72.18 ± 0.6781.15 ± 0.9283.45 ± 1.1561.18 ± 0.9974.49

Query accuracy (%), mean and SD over three matched runs. Avg. equally weights the four splits and uses unrounded per-split means. The joint-fit row is the pre-fit support row of Table R4, reported there among the weight controls. The two readouts differ only in query-answer formation, with proposals and pre-fit support held fixed.

Lower residuals. We no longer treat lower residuals on fitted context rows as evidence of better query generalisation, and drop residual magnitude from the argument in both directions. Fitting the context rows reduces their residuals by construction, so the quantity cannot separate the two readouts; the comparison is made on query accuracy alone, where the difference does not clear run-to-run variation in either readout.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.