- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes NExRI, a framework for abstract visual reasoning that constructs and executes explicit polynomial constraints on learned representations. Its IVaR module uses predictive discrepancies and recurrent verification and refinement updates to construct contextual row states. RaXO then generates polynomial hypotheses from complete context rows, evaluates their agreement on the other row, and jointly fits a shared constraint before scoring candidate completions. Training differentiates through this fitting procedure. The paper reports results on RAVEN-family benchmarks, eight PGM regimes, CLEVR-Matrices, and two Bongard benchmarks. Ablations investigate the major modules, hypothesis weighting, and readouts trained on frozen representations, while additional diagnostics examine coefficient transfer and fitting behavior.
Critical strengths
The induced constraints are part of the prediction mechanism and can be independently re-executed. RaXO derives its functions, references, and support weights from complete context rows and keeps them fixed when evaluating candidates. This provides a concrete approah for inspecting and intervening on the computations used to make predictions. The regularized fitting formulation also permits joint training with the visual representation, making this a useful architectural contribution.
The improvements can extend to meaningful generalization settings with useful component controls. On the four A-I-RAVEN splits, NExRI improves over DSRF-L by 2.3–6.5 percentage points, increasing the average from 71.5% to 75.5%. It also improves the average across eight PGM regimes from PredRNet’s 60.4% to 65.4%. The frozen-representation comparison reports 75.1% for RaXO versus 69.6% for the neural scorer, which provides evidence that the gains may involve the reasoning readout as well as representation learning.
Critical weaknesses
-
Table 3 reports 97.3% on Bongard-LOGO and 96.8% on Bongard-OpenWorld, which substantially exceeds the listed baselines. However, Appendix D.1 gives only a high-level description of the adaptation. It does not define how positive and negative support examples and an individual query are mapped to IVaR’s three-position input, how support labels enter polynomial induction and fitting, or how execution costs produce a binary prediction.
-
Table 8 explicitly describes only the frozen Type diagnostic configuration, leaving the dataset-specific encoder, pretraining, training budget, and model-selection procedures for these main results unclear.
-
The central explicit-rule contribution needs more precise positioning and validation. The broad novelty claims should account for PrAE (jointly trains perception and rule-based reasoning) or ALANS (performs instance-specific operator fitting and execution with joint training).
-
Figure 2 names the learned views Number/Position/Type/Size/Color, and Table 7 transfers coefficients between designated attribute views, but the described projections and variance/covariance regularization do not specify how those semantic assignments are obtained.
-
Appendix D.1 mentions component and proxy labels, which require further explaination. The executable polynomial interface is well defined, but the evidence for transferring particular attribute rules depends on this missing correspondence.
Questions for authors
-
How are the learned views and spatial components associated with the named attributes/components used in Figure 2 and Table 7?
-
Are the component and proxy labels mentioned in Appendix D.1 used during training?
-
What distinguishes NExRI from PrAE and ALANS’s differentiable operator fitting and execution?
-
What are the inputs, architecture, capacity, and training procedure of the Neural Scorer in Table 6?
5 comments
yqXP-W2
Table 8 explicitly describes only the frozen Type diagnostic configuration, leaving the dataset-specific encoder, pretraining, training budget, and model-selection procedures for these main results unclear.
We thank the reviewer and agree. Table 8 reports frozen Type diagnostics; the revised version will state this in its caption and add the configuration summary below as Table R2.
yqXP-W2.1A
Encoder, input and optimization for Bongard-LOGO.
A ResNet-12 trained from scratch, with 128-dimensional outputs and no pretraining, on grayscale inputs. SGD with learning rate , momentum 0.9, weight decay ; batch size eight; 100 epochs. The total parameter count is 2.09M, against 8.10M for the released B-SC and ProtoNet ResNet-12, because the channel widths differ.
yqXP-W2.2A
Encoder, input and optimization for Bongard-OpenWorld.
A LAION-2B ConvNeXt-base, trainable rather than frozen, with 1,024-dimensional outputs on RGB inputs. AdamW with learning rate and weight decay ; batch size ten; 20 epochs including five warmup epochs followed by cosine decay. The parameter count is 88.1M, comparable to MetaOptNet (88.2M) and smaller than the caption-enabled ProtoNet (320.1M) and SNAIL (347.9M).
yqXP-W2.3A
Model selection.
Selection uses validation query accuracy only, computed with the same support/query organization as training. On LOGO the validation split is traversed in full with fixed queries, so every epoch is scored on the same episode set and the comparison across epochs is paired; on OpenWorld each of the three runs selects its own best validation checkpoint. Test data enter only the reported numbers. No hyperparameter was searched on either benchmark: both schedules and both optimizer settings are the official recipes [5, 6], applied without benchmark-specific tuning.
Table R2. Configuration of the Bongard experiments in Table 3. Models are trained per benchmark; the frozen Type checkpoint behind Table 8 is not involved. Parameter counts are for the full model.
| Setting | Bongard-LOGO | Bongard-OpenWorld |
|---|---|---|
| Encoder | ResNet-12, 128-d | ConvNeXt-base, 1,024-d |
| Pretraining | none | LAION-2B, fine-tuned |
| Input | , grayscale | , RGB |
| Optimizer | SGD, lr , mom. 0.9, wd | AdamW, lr , wd |
| Batch size | 8 | 10 |
| Schedule | 100 epochs | 20 epochs, 5 warmup + cosine |
| Parameters | 2.09M | 88.1M |
| Objective | label-smoothed query CE VICReg-style regularization, uniform hypothesis priors | label-smoothed query CE VICReg-style regularization, uniform hypothesis priors |
| Augmentation, caption supervision | none | none |
| Checkpoint selection | validation query accuracy, full split, fixed queries | validation query accuracy, per run |
yqXP-W3
The central explicit-rule contribution needs more precise positioning and validation. The broad novelty claims should account for PrAE (jointly trains perception and rule-based reasoning) or ALANS (performs instance-specific operator fitting and execution with joint training).
Our contributions are threefold. 1) We introduce NExRI, an end-to-end framework that unifies explicit rule modeling with neural feature learning: relations are verifiable polynomial invariants on learned states, trained jointly by differentiating through the fitting step. 2) IVaR, a recurrent reasoning block, produces contextual states that expose the latent relation through coupled predict-and-verify and verify-and-refine loops. 3) RaXO turns these states into explicit rule objects: each complete row proposes a function and a reference, validated on the other row before a common constraint is fitted and executed on candidates.
The two closest explicit-rule methods differ at the point where a candidate is produced and how it is selected. PrAE [8] abduces predefined logical rules over learned attributes, so a relation is chosen from a fixed vocabulary; NExRI instead regresses free polynomial coefficients and references on contextual row states, and the constraint defining the rule is the object the optimiser differentiates. ALANS [9] fits operators within a given space, independently per instance; RaXO validates a proposal on the row that did not generate it and retains its support as a soft weight over all four hypotheses under one fitted criterion. IVaR keeps the states comparable across rows, which is what makes that cross-row validation well defined.
yqXP-W3. Benchmark comparison
Following the reviewer's request, Table R3 reports results for PrAE and ALANS under their original RAVEN and I-RAVEN evaluation protocols [8, 9] and the benchmark authors' protocol for Mesh and the four A-I-RAVEN OOD splits [4]. Both methods score substantially lower on the held-out splits under the benchmark authors' protocol. ECo-NSR is cited as related work but does not supply a full-system row, since its paper states that end-to-end inference is not yet available [3].
Table R3. Accuracy (%) on RAVEN-family benchmarks. See the table note for evaluation protocols.
| Method | RAVEN | RAVEN-FAIR | I-RAVEN | I-RAVEN-Mesh | OOD Color | OOD Position | OOD Size | OOD Type | Avg. |
|---|---|---|---|---|---|---|---|---|---|
| PrAE (2021) | 65.03 | 88.30 | 77.02 | 33.20 ± 0.40 | 47.90 ± 0.90 | 68.20 ± 3.30 | 41.30 ± 1.80 | 37.00 ± 1.70 | 57.24 |
| ALANS (2022) | 74.40 | 90.31 | 78.50 | 15.90 ± 2.60 | 15.20 ± 1.40 | 16.00 ± 1.00 | 23.30 ± 6.50 | 19.00 ± 3.40 | 41.58 |
| NExRI (Ours) | 99.6 | 99.7 | 99.7 | 95.1 ± 0.8 | 73.1 ± 0.9 | 82.1 ± 0.7 | 84.5 ± 0.9 | 62.1 ± 1.1 | 87.0 |
RAVEN and I-RAVEN entries are as reported in [8, 9], with PrAE's RAVEN-FAIR result from [10]. Mesh and A-I-RAVEN baselines are mean and SD over three seeds under the shared recipe of [4]. Avg. is the mean over the eight settings.
yqXP-W4
Figure 2 names the learned views Number/Position/Type/Size/Color, and Table 7 transfers coefficients between designated attribute views, but the described projections and variance/covariance regularization do not specify how those semantic assignments are obtained.
We do not claim that the model obtains a semantic assignment, nor that its latent space is decodable into attributes. The attribute names in Figure 2 are the generator's definitions (Number, Position, Type, Size, Color), human-specified and fixed before any model is trained; no attribute label enters the forward pass, the loss, or any supervision signal, and VICReg-style variance and cross-view covariance regularization treats the five views symmetrically.
The test is not whether the model recovers human-interpretable attributes or rules, but which generator attribute drives each projection. We freeze the checkpoint and, for each attribute, apply a single-attribute edit while holding the remaining attributes and the rendering seed fixed, recording the change of each validation-standardized projection feature on validation data. Table R1 reports the resulting correspondence. Rows sum to one across the five views. On validation data, each attribute's edit identifies the view with the largest response. These five maxima identify distinct views, which we name and order by the corresponding attributes. Table 7 transfers coefficients between these labelled slots. The labelling step fixes only the slot names and does not enter training.
Table R1. View–attribute correspondence.
| Changed attribute | Number | Position | Type | Size | Color |
|---|---|---|---|---|---|
| Number | 0.77 | 0.10 | 0.02 | 0.07 | 0.04 |
| Position | 0.08 | 0.81 | 0.02 | 0.05 | 0.04 |
| Type | 0.06 | 0.05 | 0.75 | 0.09 | 0.05 |
| Size | 0.04 | 0.05 | 0.06 | 0.83 | 0.02 |
| Color | 0.03 | 0.03 | 0.01 | 0.04 | 0.89 |
Projection-feature responses to single-attribute image edits, standardized on validation data. Rows are normalized across views.
yqXP-W5
Appendix D.1 mentions component and proxy labels, which require further explaination. The executable polynomial interface is well defined, but the evidence for transferring particular attribute rules depends on this missing correspondence.
Appendix D.1 defines the component/proxy-label correspondence under augmentation: a component label records which source component each part of an augmented or composed sample is built from, and the source-component proxy label records the corresponding component in the source sample. Neither enters any training loss: both are constructed offline and are never read by the training loop, and no loss term, auxiliary head, or data-loader target consumes them. Their only role is diagnostic and bookkeeping, and they are not used for checkpoint selection or hyperparameter choice. Training optimizes answer cross-entropy plus label-free VICReg-style regularization, and no annotation beyond the answer class label reaches any loss.
The correspondence that the transfer in Table 7 depends on is the view–attribute correspondence, which is established independently of these labels and is measured in our response to yqXP-W4 (Table R1). Table 7's coefficient interventions act on fixed view/read indices defined by that frozen assignment; generator rule labels define the same-rule and different-rule groups for reporting, independently of the proxy labels.
yqXP-W1
We thank the reviewer. Appendix D.1 describes the adaptation at the level of components and leaves the interface to IVaR implicit; we agree that this is insufficient to reconstruct the method. The revised appendix will state the three mappings explicitly, and we give them here. They are also the mappings used for the results in Table 3, so the numbers are unchanged.
yqXP-W1.1A
IVaR consumes two context panels and one target panel; the adaptation changes what the panels contain, not the module. A shared attention-pooling module with two learned queries pools the six positive and the six negative supports separately, producing two anchors per class. A class's two anchors occupy the context positions, and the panel being related occupies the third. During fitting, each support row forms its own three-position input [ac1,ac2,ϕ(xic)], so a class yields six such inputs; the six are neither concatenated nor fed as six panels. During execution, the query is encoded once and paired with each class's anchors, giving [ac1,ac2,ϕ(q)] for c∈+,−; both class-conditioned inputs are processed in one batch, so the two-class decision needs no second encoder pass. Support fitting completes before any query enters execution, so query information never reaches the programs.
yqXP-W1.2A
Support labels enter induction as a partition of the fitting set, not as a supervised term in the objective. The six positive rows and the six negative rows are fitted as two independent six-row problems, so each class yields its own program and no support label appears in the loss that fits it; a positive row is related to positive context and a negative row to negative context, and the query enters neither proposals nor references. Joint Fit minimizes the mean squared residual over each class's own six rows, so the coefficients and references of the two programs are estimated from disjoint support sets. Discrimination between them is deferred to execution rather than expressed as a margin during fitting.
yqXP-W1.3A
Execution is deterministic: anchors, coefficients, references and slot weights are fixed before the query is scored. A query yields 60 execution costs per class, from 15 read/view slots and four hypotheses, each averaged over the three degrees; each cost d enters pooling as −αd, with a shared positive scale α=1+softplus(a) trained jointly with the query cross-entropy. For each class, we take a log-mean-exp over the four hypotheses, then an average over the 15 slots with normalized support-derived sigmoid weights. The resulting class evidence enters the two-class softmax as its logit, and the prediction is the argmax. No threshold or temperature is tuned and no post-hoc calibration is applied: the boundary is fixed once the programs and weights are, before any query is executed, so α is the only calibration parameter and it is trained rather than tuned on the benchmark.