- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
NBSR routes inputs through experts that add positive evidence to a Dirichlet state. The state conditions later routing and supports early exit and abstention. The paper derives concentration properties and evaluates variants on classification, synthetic language modeling, partially observable control, and diagnostic test selection.
Critical strengths
- Accumulating expert contributions provides inspectable intermediate predictions and a mechanism for adaptive computation.
- The paper reports both positive and negative results, including poorer diagnostic calibration and reduced language-modeling throughput. These results help expose the method's limitations.
Critical weaknesses
- Equation 3 has an unbounded concentration incentive.For , the predictive mean remains uniform and the classification loss remains , while the Dirichlet differential entropy tends to as . Thus, for , the objective can improve without improving predictions or acquiring information. Theorem 1 does not validate concentration as epistemic certainty. A likelihood interpretation for adding learned outputs from the same input is also missing.
- The empirical evidence is insufficient for the uncertainty and efficiency claims. The CIFAR-10 accuracy gain is 0.14% in one seed. Diagnostic ECE is worse than the MLP's, and language modeling is slower with higher perplexity. The control experiment compares NBSR's abstention with baselines required to choose movement actions, which does not isolate the benefit of its uncertainty representation. An evidential baseline is also missing.
Questions for authors
- What prevents the concentration scaling degeneracy in Equation 3? Can you specify the constrained or regularized objective and show that its uncertainty scores remain meaningful?
- How do early exit and OOD detection compare across seeds against evidential and softmax baselines with abstention tuned on a validation set?
3 comments
Thank you for the acknowledgment of the concentration degeneracy and the clarification of the Bayesian interpretation. The log-score formulation addresses the unbounded-loss issue in principle. However, the revised manuscript states that this formulation has not been evaluated, and the reported CIFAR results retain the original objective. Proper scoring of predictive means also does not imply meaningful concentration-based uncertainty. The requested evidential baseline, and validation-tuned abstention/OOD comparisons remain future work. I therefore appreciate the corrections but remain unconvinced that the current evidence establishes the proposed benefits.
Thank you for the follow-up, and for acknowledging the corrections to the objective and Bayesian interpretation. We agree that the proposed bounded-evidence formulation has not yet been experimentally evaluated, and that proper predictive scoring does not by itself guarantee calibrated concentration-based uncertainty.
Regarding the objective, the revised manuscript explicitly identifies the unbounded direction in the original entropy-regularized loss and introduces a mathematically well-posed alternative using bounded evidence and proper intermediate log scores (Sec. 3). Importantly, this issue concerns the training objective rather than the underlying sequential routing and Dirichlet evidence-update mechanism, which remains applicable with the revised formulation. We have also clarified that the existing CIFAR results use the original objective and therefore do not establish the effectiveness of the replacement.
Regarding empirical evidence, we agree that matched evidential baselines and validation-tuned abstention/OOD evaluations are needed to establish the individual benefits of the proposed mechanisms. Nevertheless, the CIFAR-10 results (96.74% accuracy, 0.015 ECE) and adaptive-depth evaluation (nearly 90% first-stage exits at unchanged observed accuracy) provide encouraging initial evidence for the feasibility of sequential belief-conditioned computation, although they do not establish superior uncertainty calibration or computational efficiency.
We believe the methodological contribution remains useful despite these limitations. NBSR integrates hard expert routing, evolving predictive beliefs, additive evidence accumulation, and adaptive computation into a unified sequential inference framework. The revised mathematical formulation addresses the identified objective defect in principle, while the existing experiments demonstrate the feasibility of the architecture across several settings. Although further controlled evaluations are needed to establish its full benefits, we believe these contributions offer a valuable direction for neural inference, and outweigh the limitations of the current experimental validation.
Thank you for the thoughtful review. We clarify guarantees, architectural contribution, and propose revisions.
core contributions and novelty. NBSR makes three contributions: (a) Belief-conditioned hard routing: a persistent oracle hx, an evolving Dirichlet state αt, and routers selecting one expert based on [hx,αt], with additive evidence updates αt+1=αt+et (Secs. 3–4). Straight-through Gumbel-Softmax enables end-to-end training of discrete routing. (b) Explicit intermediate beliefs: each visited expert contributes model-generated evidence, yielding inspectable prediction trajectories, path-wise attribution, and uncertainty-based stopping/abstention (Figs. 1–3). (c) Sequential decision extensions: NBSR-Mem incorporates recurrent memory for partially observed control, while AR-NBSR performs sequential, cost-aware diagnostic panel selection (Secs. 5.4–5.5). The novelty lies in integrating these mechanisms into a shared inference framework, not in Dirichlet conjugacy or Gumbel-Softmax individually.
Concentration degeneracy and mathematical correction. We agree with the counterexample: for α=c1, the predictive mean remains uniform, the NLL is logK, but H(Dir(α))→−∞ as c→∞. Thus Eq. (3), with λ>0 and unrestricted evidence, is unbounded below. Thm. 1 guarantees monotone concentration growth and a decreasing upper bound on variance, not calibrated epistemic uncertainty or information gain. We propose replacing the entropy penalty with intermediate proper log scores and bounded evidence: et,k=Emaxsigmoid(gt,k), mt=αt/∑kαt,k, and L=−logmT,y+λ∑t<T[−logmt,y]+γ∑tc(vt). For finite depth, total concentration is bounded by K+tKEmax, while intermediate log scores reward early correct prediction rather than concentration inflation. Notably, medical NBSR uses λ=0, language omits the entropy penalty, and NBSR-Mem employs target-masked evidence regularization after the original penalty caused optimization difficulties (Sec. 5.4). We will update the objective and prioritize comparisons against λ=0, reporting evidence scales, calibration, and early-exit behavior under the revised formulation.
Bayesian interpretation and uncertainty guarantees. The update in Eq. (2) is exact Dirichlet pseudo-count algebra: expert outputs can be represented by generalized-likelihood factors ∏kpket,k(x). However, successive experts often query the same oracle, so their evidence is not automatically equivalent to conditionally independent observations or a calibrated Bayesian posterior. Thm 2 establishes recovery of the Bayes-optimal predictive mean only under idealized capacity, optimization, and λ=0 assumptions. We will distinguish the mathematical belief-update mechanism from probabilistic calibration, and avoid interpreting growing concentration alone as proof of genuine epistemic certainty.
Existing empirical evidence and limitations. On CIFAR-10, NBSR achieves 96.74% accuracy / 0.015 ECE, compared with 96.60% / 0.026 for flat ResNet-18 and 96.43% / 0.027 for sparse MoE (Table 1). Fig. 2(c) also shows separation in terminal entropy between CIFAR-10 and SVHN. These support the architecture, but do not establish calibrated epistemic uncertainty or reliable OOD detection. The current CIFAR comparison uses one seed, without OOD AUROC or matched selective-risk evaluation. Medical ECE is worse than the MLP (0.166 vs. 0.016; Table 2), and language modeling is slower with slightly higher perplexity. We do not claim improvements across every task. We will prioritize multi-seed evaluation, confidence intervals, and calibration/OOD comparisons after correcting the entropy objective.
Early exit, OOD abstention, and fair baselines. CIFAR Fig. 2(b) demonstrates that nearly 90% of samples exit after one expert without reducing accuracy, supporting adaptive expert computation. However, total inference time remains comparable to the flat model (5.1 s vs. 5.0 s, Table 1), so this is not yet an end-to-end speedup. In POMDP navigation, NBSR-Mem and CNN-GRU both achieve 100% success, while NBSR-Mem abstains on all evaluated OOD hazards (Table 4). We agree that the baselines were not given an equivalent abstention option, making their 0% halt rate an unfair safety comparison. We will prioritize validation-tuned reject options for flat softmax, one-pass evidential, and MoE baselines, evaluating risk–coverage, OOD AUROC/FPR, ID false-rejection, and accuracy–latency tradeoffs across seeds. These tests will establish whether NBSR's evolving belief offers benefits beyond ordinary prediction confidence.
Overall, the mathematical issue concerns the unrestricted entropy regularizer, not the feasibility of belief-conditioned sequential routing itself. We will revise the objective and its theoretical interpretation.