acceptodds
Under review as a conference paper at ICLR 2027

Linear Time Generative Inference in Neural Probabilistic Circuits

Abstract

Neural Probabilistic Circuits (NPCs) are neuro-symbolic models that combine both the recognition capacity of modern neural networks and the tractable reasoning of probabilistic circuits. In NPCs, a neural network predicts human-interpretable attributes from the input, and a follow-up probabilistic circuit models the joint distribution of those attributes and the class label. To connect the two modules, the existing inference procedure in NPCs averages the circuit’s class conditional probabilities over all attribute configurations, both during training and at test time. In the worst case, the average operation costs time exponential in the number of attributes, which limits the scalability of NPCs to applications with at most tens of attributes. To overcome the above limitation, we propose a new inference procedure, which we term as the generative inference, that averages each class by the joint probability, rather than the class conditional probability, over the network’s attribute predictions. We show that under factorized attribute predictions from the neural network module, each smooth and decomposable circuit can instead compute the overall prediction exactly in one forward pass of the circuit, which reduces the inference time complexity from exponential in the number of attributes to linear in the circuit size. Our key insight that enables the exponential reduction lies in the observation that factorized functions over the leaf nodes in a smooth and decomposable circuit can be absorbed into the leaf factors, hence effectively exploiting the tractability of circuits. Empirically, the generative inference helps lift NPCs to datasets with hundreds of attributes, which the original discriminative inference cannot scale to. In terms of statistical performance, by reusing the generative inference during learning, we demonstrate that the learned NPCs are at least comparable to, and sometimes better than the original NPCs.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
probabilistic circuits, tractable probabilistic inference, neuro-symbolic AI, concept bottleneck models, neural probabilistic circuits, soft evidence, interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_MnWrOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a method for dealing with an intractability that arises in Neural Probabilistic Circuits (NPCs) at the connecting point of a neural network attribute extractor and the probabilistic circuit modelling the joint distribution of those features with a class label variable. To compute the conditional class probabilities given the data traditionally used for classification, it is required to marginalize over the space of attributes, the computational complexity of which grows exponentially in the number of attributes. Instead of computing the class conditional, the authors propose a method for computing a proxy class score based on evaluation of the class-attribute joint distribution, and which leverages the structural properties of the circuit to compute class scores in time linear w.r.t. the size of the circuit. The authors provide a bound for the error their approximate inference method induces, and compare their method with comparable SotA methods as well as the ground truth posterior where possible. The proposed method allows for at least an order of magnitude larger attribute spaces to be used, maintains similar performance to the ground truth in classification accuracy, and achieves performance comparable to SotA methods whilst having the added tractability of using a circuit model.

Critical strengths
  • S1: The paper rigorously presents and justifies an approximate inference method that provides substantial efficiency gains in inference in NPCs, taking complexity from exponential to linear. This has a concrete outcome of allowing for NPCs to be used in tasks with substantially larger attribute sets, and hence data sets that were previously out of reach. Empirical results show that the approximation introduced by the method maintains similar performance to its exact counterpart and to SotA competitors, whilst still giving the added benefits of a circuit based model.
  • S2: The authors derive a bound for the error that their generative inference method produces w.r.t. the true class posterior giving a guarantee for performance, and also a pointwise diagnostic for where the error is greatest.
Critical weaknesses
  • W1: The paper does not have a limitations or discussion section, and does not discuss future work. In particular, it would be useful to have further discussion (and perhaps analysis) on the error bound that is derived and how this behaves empirically.
  • W2: The paper would also benefit from discussing and analyzing other limitations that not evaluating the class conditional directly induces. For example, other performance metrics such as NLL or ECE that give insights into uncertainties in predictions could be shown.
Questions for authors
  • Q1: How does the generative inference method proposed compare to the discriminative inference methods in terms of other inference tasks, such as those involving uncertainty quantification?
  • Q2: What are the next steps for inference in NPCs? What are limitations that you might work on next, or applications that NPCs might be applied to with the added flexibility in attribute size? How could, for example, the performance gap between NPC models with CBMs and DNNs be further reduced?
  • Q3 I did not find the "DNN" model shown in Table 1 specified in the main text. What exactly is this model, and how does it relate to the CBM and NPC models?
  • Q4: Given that the model underperforms in terms of accuracy w.r.t. CBMs and DNNs (AwA2, CelebA, CUB200, SUN), what are some of the reasons practitioners might use an NPC instead?
  • Q5: It is interesting to see the effects of the generative and discriminative (exact and sampled) inference methods disentangled in the training vs. inference stages. For example, using a model trained on the generative objective but evaluated on the discriminative objective and vice versa. Is this the experiment discussed in section 6.3, and what was the overall conclusion?
3 comments

We thank the reviewer for the constructive suggestions. We appreciate your positive assessment of the efficiency gains, experiments with larger attribute sets, and theoretical analysis.

W1. Discuss limitations, future work, and the bound empirically.

Section 5 explains diagnosis on annotated data and a sufficient condition after full correction. New Appendix F evaluates both the original bound and a tighter form using an annotation-based reference. Both hold on all evaluated inputs. The original bound matches the mean total-variation error on MNIST-Add and GTSRB, but exceeds one in mean on the other four. The tighter form is also below one in mean on AwA2 and CUB-200; Appendix F explains why (Table 7). Error detection without test annotations remains future work. Appendix H discusses accuracy, calibration, and training–inference mismatch; Section 7 summarizes the next steps.

W2 and Q1. Report NLL or ECE and compare predictive uncertainty.

New Appendix H, Table 11, reports expected calibration error (ECE), which measures the gap between confidence and accuracy, before and after validation-fitted temperature scaling. With training and inference matched, calibrated ECE differs by less than 1.6 percentage points on five benchmarks. SUN is the exception: 15.27% for generative inference versus 2.90% for discriminative inference. Before calibration, generative probabilities are underconfident on both CUB-200 and SUN. Omitting the learned training scale from the reported probabilities may contribute to this underconfidence.

Q2. What next steps could improve accuracy or use the larger attribute sets?

In Appendix H and Section 7, we propose testing whether training the circuit on annotated attributes as well as recognizer predictions, and learning its structure, improve classification and correction. With larger attribute sets, the tractable attribute posteriors could also help choose which missing attribute to request from a user.

Q3. What exactly is the DNN in Table 1?

We adopt the DNN and joint-CBM baseline designs from Chen et al. (2025). The DNN has a linear class head and no attribute layer; the joint CBM applies a linear class head to attribute logits. Both use the same ResNet backbone as NPC: ResNet-34 on the first four datasets and ResNet-50 on CUB-200 and SUN. Section 6.1 and Appendix D give the configurations.

Q4. Why use an NPC despite lower accuracy?

NPC offers these capabilities while maintaining accuracy close to the baselines on several benchmarks (Table 1):

  • Interpretability and prior knowledge. NPCs and CBMs expose named attributes for inspection and correction; our DNN baseline does not. NPC’s circuit can also encode logical rules, as demonstrated by Chen et al. (2025, Section 3.2.2). The standard CBM head has no corresponding logical component.

  • Missing attributes and corrections. The circuit is a generative model, so it can sum over missing attributes and condition on supplied values. A standard CBM’s deterministic head requires missing inputs to be filled. New Table 9 compares NPC with joint, sequential, and independent CBMs using all predicted attributes, half withheld, or all corrected. Corrections use each model’s training targets and Koh et al.’s (2020) percentile replacements for the joint and sequential CBMs; Appendices D and G give the details. NPC outperforms the joint CBM after full correction on five benchmarks, while the joint CBM does better with half the groups missing on five; CelebA reverses both comparisons. On CUB-200, the joint CBM is best in the first two settings, but full correction gives 72.5% for NPC and 1.7% for the joint CBM. Sequential and independent CBMs have higher accuracy after full correction, but lower accuracy than both NPC and the joint CBM with all predicted attributes available and with half the groups missing. NPC therefore offers a favorable tradeoff across these three settings.

  • Attribute inference without images. Given half the annotated attributes, NPC predicts the rest better than a baseline that always predicts the most frequent training value on five of six benchmarks. The gains in balanced accuracy range from 4.0 to 25.4 percentage points across these five benchmarks (new Table 10). Both methods are near chance on MNIST-Add, whose digits are independent.

Our contribution makes these NPC queries tractable at larger attribute counts.

Q5. Does Section 6.3 separate training from inference, and what does it show?

Yes. Table 6 in Appendix E applies each feasible inference rule to each trained model across all six benchmarks; Table 1 compares matched training and inference. Switching rules can reduce accuracy in either direction. This supports training with the inference intended for use; the best pair depends on the dataset (Section 6.3).

Thank you for addressing my questions and concerns, and for adding the further experiments and sections to the paper.

The authors address and demonstrate an effective and low cost solution (fine-tuning) for the scenario of mismatched training and inference that reviewer 7Ykd identifies (W1). The authors also improved the method's computational efficiency based on reviewer 1NBq's observation in W2. The main weakness seems to be 1NBq's observation regarding the usefulness of the bound derived, which the authors address with new derivations and experiments.

The method proposed provides a clear contribution to NPCs in allowing these models to be applied to tasks with large attribute sets (for example, CUB-200 and SUN). The method outperforms a baseline approximation method in these cases.

Thank you for your careful review and thoughtful follow-up! We appreciate your recognition of the contribution and your constructive feedback, which has helped us strengthen the paper. Please let us know if there are any remaining questions or points we can clarify.

Reviewer_1NBqOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes an alternative prediction rule for Neural Probabilistic Circuits (NPCs). Whereas the earlier NPC formulation of Chen et al. (2025) averages the circuit’s class-conditional probabilities under the neural attribute predictions, the proposed method averages joint class–attribute scores and normalizes afterward. This reformulation produces a factorized soft-evidence query that can be evaluated exactly using a smooth, decomposable circuit. This predictor has been used to train NPCs and also to provide tractable inference.

Critical strengths

1- The proposed predictor avoids enumerating attribute assignments and provides tractable inference under circuit tractability assumptions, which is helpful to scale the NPC to a high-dimensional attribute space.

2- The authors clearly position the paper relative to more recent literature on probabilistic circuits.

Critical weaknesses

1- Relative to the classical arithmetic-circuit literature (Darwiche 2003), I do not see a substantive new inference mechanism. The resulting computation is a standard factorized soft-evidence marginal query, and the authors acknowledge that its tractability follows from existing results.

2- Established AC inference methods can recover the scores for all states of the class variable simultaneously using one upward and one downward traversal, with total cost linear in the circuit representation. Their applicability to smooth, decomposable SPNs is also established. However, Algorithm 1 does not exploit this existing capability, and it repeats the circuit evaluation for each class,

3- Theorem 2 in the current form is not very informative. Both terms in the bound depend on true distribution quantities, and no estimator has been provided to actually estimate the error bound.

4- Both the DNN and CBM baselines have higher mean classification accuracy than the proposed method on four of the six datasets. Improvements over sampled NPC inference are useful, but they do not by themselves establish why the resulting model is preferable to these simpler alternatives. The benefits of NPC should be demonstrated experimentally rather than only be justified based on properties of PCs.

Darwiche A. A differential approach to inference in Bayesian networks. Journal of the ACM (JACM). 2003 May 1;50(3):280-305.

Questions for authors

1- "The bound is pointwise, so it can flag individual inputs on which the composition is untrustworthy, and both terms have computable surrogates on a trained model." please provide the details of the surrogates and how the bound is used to flag untrustworthy composition. In general, I would like to know what usable guarantee Theorem 2 provides.

2- Can you characterize or measure the effect of reweighting the neural attribute posterior by the circuit’s attribute marginal, including its impact on calibration and class decisions?

3- What demonstrated benefit justifies using the proposed NPC instead of the DNN or CBM baselines?

7 comments

We thank the reviewer for the detailed review. We appreciate your positive comments on scalability and our discussion of related work.

W1. No new inference mechanism relative to Darwiche (2003).

We agree that computing soft-evidence probabilities is established work (Darwiche, 2003), and we do not claim it as our contribution. Our contribution is identifying a generative approximation to NPC’s original inference that can be computed as a soft-evidence query, replacing exponential enumeration with computation linear in circuit size. We have clarified this in Contribution (1) and the discussion after Theorem 1.

W2. Algorithm 1 repeats the circuit evaluation for each class.

Thank you for pointing this out! This helped improve the computational complexity of Algorithm 1. It now obtains all class probabilities under the supplied soft evidence and the attribute marginals using one forward and one backward pass, in time linear in circuit size. We have updated Algorithm 1 in Appendix C and explained the computation in Theorem 1 and Appendix B.

Our experimental EiNets already return all class scores in one forward pass because the class enters at the root layer. The experimental computation, accuracy, and timings are therefore unchanged.

W3 and Q1. How can Theorem 2 be evaluated, what guarantee does it give, and can it flag errors?

We clarify two uses of Theorem 2 in Section 5 and new Appendix F: diagnosing errors on annotated data and giving a guarantee after full correction. We agree that its unknown data distributions prevent computing it from a test image alone.

  • What can annotated data tell us? With adequate annotation coverage, estimates can guide inspection of attribute inference or the circuit’s class conditional. The attribute term includes circuit reweighting, so it does not identify which module needs more training. Table 7 uses a reference built from attribute annotations and label frequencies, satisfying attribute sufficiency. Both bounds hold on all evaluated inputs. The original bound matches the mean total-variation error on MNIST-Add and GTSRB (0.011 and 0.001), but its mean exceeds one on the other four. The tighter version, which measures attribute differences after mapping them to classes, is below one in mean also on AwA2 (0.771) and CUB-200 (0.763). Appendix F explains both the improvement and the limits imposed by annotation coverage.

  • When does correction guarantee a correct prediction? If an image has a single true attribute vector that also determines its class, fully correcting the attributes removes the attribute-error term. A circuit error below 1/21/2 then guarantees the correct class. Table 8 reports the threshold for all five benchmarks with single-vector replacement targets; AwA2 permits multiple assignments. Every input satisfies both conditions on MNIST-Add and GTSRB. Only subsets pass the error threshold on CelebA, CUB-200, and SUN, and label uniqueness is also required. The guarantee therefore depends on both the supplied attributes and the circuit’s class conditional.

  • Can we flag errors without test annotations? The pointwise bound alone does not provide a usable error flag. Appendix F explains how validation label frequencies give a partial circuit-error score. This score omits unseen assignments and does not estimate attribute error, so a low value need not indicate a reliable prediction. We have removed the general error-flagging claim and identify reliable estimation of both bound terms as future work.

W4 and Q3. Why choose NPC over DNN and CBM baselines? Demonstrate its benefits.

NPC offers these capabilities while maintaining accuracy close to the baselines on several benchmarks (Table 1):

  • Interpretability and prior knowledge. NPCs and CBMs expose named attributes for inspection and correction; our DNN baseline does not. NPC’s circuit can also encode logical rules, as demonstrated by Chen et al. (2025, Section 3.2.2). The standard CBM head has no corresponding logical component.

  • Missing attributes and corrections. The circuit is a generative model, so it can sum over missing attributes and condition on supplied values. A standard CBM’s deterministic head requires missing inputs to be filled. New Table 9 compares NPC with joint, sequential, and independent CBMs using all predicted attributes, half withheld, or all corrected. Corrections use each model’s training targets and Koh et al.’s (2020) percentile replacements for the joint and sequential CBMs; Appendices D and G give the details. NPC outperforms the joint CBM after full correction on five benchmarks, while the joint CBM does better with half the groups missing on five; CelebA reverses both comparisons. On CUB-200, the joint CBM is best in the first two settings, but full correction gives 72.5% for NPC and 1.7% for the joint CBM. Sequential and independent CBMs have higher accuracy after full correction, but lower accuracy than both NPC and the joint CBM with all predicted attributes available and with half the groups missing. NPC therefore offers a favorable tradeoff across these three settings.

  • Attribute inference without images. Given half the annotated attributes, NPC predicts the rest better than a baseline that always predicts the most frequent training value on five of six benchmarks. The gains in balanced accuracy range from 4.0 to 25.4 percentage points across these five benchmarks (new Table 10). Both methods are near chance on MNIST-Add, whose digits are independent.

Our contribution makes these NPC queries tractable at larger attribute counts.

Q2. How does reweighting affect calibration and class decisions?

New Appendix H isolates the effect of reweighting by applying both inference rules to models trained with generative inference, across all six benchmarks.

  • Accuracy and decisions. Generative class accuracy is higher on AwA2, CUB-200, and SUN, and close to discriminative inference on the other three benchmarks (Table 6). Reweighting leaves mean attribute accuracy unchanged or higher to the reported precision (Table 12). The added bound shows that changes in class probabilities cannot exceed the change in the full attribute distribution, measured in total variation (Eq. (21)).

  • Calibration. Expected calibration error (ECE) measures the gap between confidence and accuracy. On these checkpoints, generative ECE is similar or lower after validation-fitted temperature scaling on five benchmarks. SUN is the exception: 15.27% versus 4.66% (Table 11). Before calibration, generative probabilities are underconfident on SUN. The learned log-score scale used in training is omitted from the reported probabilities, which may contribute to this underconfidence.

W3 and Q1 (follow-up). Can the decomposition show which module causes the error?

The reviewer asked for an estimator of the bound and for the usable guarantee it provides. Below, both terms are computed from annotated examples with two forward passes each, their averages over nn examples are within log⁡(2/δ)/(2n)\sqrt{\log(2/\delta)/(2n)} of their expectations with probability 1−δ1-\delta regardless of the number of attributes, and what they provide is a measured answer to whether the recognizer or the circuit is responsible for the error.

Here we add a second form of the decomposition that evaluates the circuit at the true attributes. The circuit-side values use only results already reported in Tables 7 to 9.

Why can Theorem 2's circuit term be misleading? Theorem 2 and its tighter form weight the circuit error by qω(a∣x)q_\omega(a\mid x). Where qωq_\omega puts its mass on attribute vectors with no annotations, Appendix F sets the circuit error to zero, so the term can be small even when the circuit is inaccurate. On CUB-200, qωq_\omega places 0.1% of its mass on the annotated support (Table 7), and the circuit term is 0.001.

What does the second form measure? Applying the triangle inequality through the circuit's prediction from the true attributes a⋆a^\star gives, for every annotated example (x,a⋆,y)(x,a^\star,y),

1−hgen(y∣x)≤∥hgen(⋅∣x)−Pω(⋅∣a⋆)∥TV⏟effect of attribute errors on the class prediction+1−Pω(y∣a⋆)⏟circuit error at the true attributes.1-h^{\mathrm{gen}}(y\mid x) \le \underbrace{\lVert h^{\mathrm{gen}}(\cdot\mid x)-P_\omega(\cdot\mid a^\star)\rVert_{\mathrm{TV}}}_{\text{effect of attribute errors on the class prediction}} + \underbrace{1-P_\omega(y\mid a^\star)}_{\text{circuit error at the true attributes}}.

Both terms compare distributions over classes, so neither approaches one as the number of attributes grows. In 0-1 error the same split reads Rgen≤Rshift+RcircR_{\mathrm{gen}}\le R_{\mathrm{shift}}+R_{\mathrm{circ}}, where RgenR_{\mathrm{gen}} is the error rate, RcircR_{\mathrm{circ}} the error rate after full attribute correction, and RshiftR_{\mathrm{shift}} the fraction of predictions that full correction changes.

What do the results show? The circuit side is already measured:

RgenR_{\mathrm{gen}}RcircR_{\mathrm{circ}}Circuit term, Theorem 2 (Table 7)Circuit error at a⋆a^\star (Table 8)
MNIST-Add1.0%0.0%0.0010.001
GTSRB0.1%0.0%0.0000.000
AwA212.8%18.3%0.446n/a
CelebA65.4%32.9%0.5240.593
CUB-20020.8%27.5%0.0010.715
SUN67.8%94.9%0.2650.978

On AwA2, CUB-200 and SUN, Rcirc>RgenR_{\mathrm{circ}} > R_{\mathrm{gen}}: the circuit is the bottleneck, and supplying correct attributes does not help. On CUB-200, Theorem 2's circuit term (0.001) suggests the opposite, while the circuit error at the true attributes is 0.715. On CelebA, correction halves the error, so the attribute side dominates. On MNIST-Add and GTSRB both sides are negligible.

These annotated validation data identifies which module to improve, and on CUB-200 it corrects the conclusion that Theorem 2's terms would give.

What are its limits? With labels the error itself can be measured, so this is not a tighter estimate of the error; its use is to show which module the error comes from. Since the circuit is trained on the recognizer's outputs, the two terms can partly cancel, and their sum can be loose. RshiftR_{\mathrm{shift}} and the attribute term will be reported in the final version.

Reviewer_7YkdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed a more computationally efficient (but with non-negligible performance degradation) inference approach to Neural Probabilistic Circuits (NPC), and then reuse the generative inference during learning to mitigate such performance degradation.

Critical strengths
  • The proposed inference approach is truly computational efficient
  • Some sort of performance guarantee is provided
Critical weaknesses
  • If we simply apply the proposed approach on a pre-trained NPC, the performance degradation is sometimes very significant (see AwA2 in Table 6)
  • However, if we reuse the proposed approach during training, the statistical meaning of the learned probabilistic circuit becomes unclear. We already see that NPC (and its variant) generally performs worse than CBM regarding classification accuracy (Table 1). This in itself is not a problem, as the paper said:

This circuit gives an NPC exactly what the head in existing CBMs lacks: it scores a class from the attributes’ joint behavior, so it allows scoring a class under partial evidence and inferring the unobserved attributes.

But once the proposed approach is also used in inference, the training objective deviates from the original one that fits the joint distribution. In such a case, is the aforementioned merits of NPC still valid?

Questions for authors

See "weaknesses"

6 comments

We thank the reviewer for the helpful questions. We appreciate your recognition of the computational efficiency and theoretical guarantee.

W1. Generative inference can substantially reduce a pretrained NPC’s accuracy.

We agree that the drop matters when reusing a pretrained NPC. Our inference approximates the original predictor, so switching rules after training introduces a mismatch between training and testing. Table 6 in Appendix E already reports this effect: on AwA2, the enumeration-trained model falls from 89.44% to 79.09% with our inference, while our model falls from 87.15% to 82.11% under enumeration.

In new Appendix H, we test whether brief fine-tuning can recover the mismatch loss without repeating pretraining. AwA2 has the largest such drop among the benchmarks where enumeration is feasible. Ten epochs with our objective, compared with 150 for a full Stage 3, raise mean generative accuracy from 79.09% to 87.02% across three AwA2 models. This is close to the 87.15% obtained with our objective from the start. On this benchmark, fine-tuning recovers most of the loss without repeating Stages 1 and 2.

W2. What do the circuit and recognizer represent after Stage 3? Do partial evidence and attribute inference still work?

After normalization, the circuit still defines a joint distribution over attributes and class, fine-tuned for classification. Generative inference conditions this joint on the recognizer’s soft evidence. Its structure still supports exact marginalization and conditioning (Remark 1). We agree, however, that a valid joint model need not fit the data well, so we added measurements of both fit and usefulness.

  • Recognizer. Stage 3 retains attribute supervision. Across all six benchmarks, mean attribute accuracy changes by less than two percentage points from Stage 1 (Appendix E, Table 6). Thus the recognizer continues to predict the supervised attributes with similar accuracy after class fine-tuning. These accuracy measurements do not establish calibrated attribute probabilities.

  • Joint-model fit. New Appendix H, Table 13, compares probabilities assigned to held-out annotations before and after Stage 3. Joint log probability is nearly unchanged on MNIST-Add, GTSRB, and AwA2, improves on CelebA, and falls on CUB-200 and SUN. Thus annotation fit is stable or improved on four benchmarks and worse on two. The decrease is larger than with the sampled objective on CUB-200 and smaller on SUN. This does not establish closeness to the true joint distribution.

  • Partial evidence and missing attributes. With half the groups withheld, class accuracy improves on AwA2, CUB-200, and SUN, and stays within about one percentage point of Stage 2 on MNIST-Add, GTSRB, and CelebA (Table 13). Without an image, the circuit outperforms a baseline that always predicts the most frequent training value on five benchmarks, with gains of 4.0 to 25.4 percentage points in balanced accuracy; both are near chance on MNIST-Add, whose digits are independent (new Appendix G, Table 10). The joint therefore remains useful for both queries after fine-tuning across these tests.

Thank you for the additional experiments. One more question: If the primary goal is classification, a standard CBM is computationally efficient and generally achieves higher accuracy. If the primary goal is to "score a class under partial evidence and infer the unobserved attributes", the original NPC approach already supports it, and the proposed approach does not perform better. Then, what are the potential applications of this method, for which "the proposed approach > CBM + NPC"?

Thank you for the question. NPCs target applications requiring both neural classification and probabilistic reasoning about attributes. For example, in expert assisted recognition, a user may supply or correct some attributes, and the model must update its class prediction and infer the remaining attributes. Standard CBMs provide efficient classification, but do not model the joint distribution of attributes and classes that supports these additional NPC queries.

However, prior NPC methods face an exponential enumeration cost that limits their practical use to small attribute spaces. Previous NPC image classification studies report at most eight attribute groups. Our generative inference replaces this enumeration with computation linear in circuit size, enabling efficient prediction and joint training while retaining NPC’s probabilistic queries. This allows us to use datasets such as CUB-200 and SUN with approximately 100 attributes. Compared with a sampled approximation to the original NPC predictor, our method improves classification accuracy by 10.01 and 18.47 percentage points, respectively. On CUB-200, it achieves 79.18%, close to the joint CBM’s 81.75%.

Our additional experiments demonstrate the retained benefits. NPC achieves higher accuracy than the joint CBM after full attribute correction on five of six benchmarks, and supports attribute inference without images. The contribution is therefore to make NPCs practical at attribute scales that CBMs already handle, without giving up the probabilistic reasoning capabilities that motivate using NPCs.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Request for reviewer follow-up

Dear Reviewers 7Ykd, 1NBq and MnWr,

A gentle reminder that we have responded to your reviews. We would be happy to follow up on any remaining concerns, and would welcome any additional feedback. We thank you for your valuable time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.