- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TRNK, a method for approximate inference and parameter learning in discrete Bayesian networks with a supplied graph structure. It constructs overlapping triplet and four-variable regions, uses a neural network to predict local probability distributions, and enforces agreement between overlapping regions. The method is evaluated on fixed-parameter inference and learning conditional probability tables from incomplete data.
Critical strengths
1- The paper addresses an important computational problem. Repeated inference over missing variables is a major part of learning Bayesian-network parameters from incomplete data. Sharing an inference network across records is a useful approach to reducing this cost.
2- The consistency construction is a meaningful contribution. Rather than relying only on a penalty to encourage agreement between overlapping regions, the method builds this agreement into its parameterization. Proposition 2 provides a formal result for the specified consistency constraints.
3- The experiments cover several network families and include relevant baselines. The paper reports inference comparisons on eight networks and parameter-learning comparisons on sixteen. The reported results show lower validation prediction error on twelve of sixteen networks and lower probability-table error on fifteen of sixteen. These are promising results, although the evaluation limitations below affect their interpretation.
4- Some comparisons help isolate the proposed components. IJGP-neural uses the same inference solver with a different region construction, and Appendix E.2 compares the previous TRC construction with TRI. These are useful controls rather than comparisons against unrelated methods alone.
Critical weaknesses
1- There is no independent test set for the reported parameter-learning prediction results. Appendix E.1.3 states that hyperparameters are selected using validation data, with no separate test set. The same validation criterion is also used for checkpoint selection and reporting. This may make the reported prediction results optimistic. The comparison against the known generating probability tables remains informative, but it does not replace independent evaluation of predictive performance.
2- The robustness of the numerical results is insufficiently established. The reported experiments use fixed seeds, and the tables do not show variation across repeated training runs or independently generated datasets. The incomplete-data experiments use a missingness probability of 0.3, and Appendix B.1 states that all original variables in the experiments are binary. Results across different missingness levels and non-binary variables would better support the general claims.
3- The efficiency comparison needs more qualification. In the large-data experiment, TRNK uses five A800 GPUs, while VMP runs on a CPU. Region-construction time is excluded, and some VMP results are taken before convergence because the time budget is reached. These comparisons demonstrate performance under the stated implementations, but they do not isolate the computational benefit of the algorithm itself. End-to-end runtime, memory use, and resource-matched comparisons would strengthen the efficiency claims.
4- The connection between the theoretical analysis and the implemented region construction needs clarification. The orthogonality result concerns a specific complete-DAG construction, while the implementation uses a different parent-grouping description. Furthermore, the local approximation result does not establish improved accuracy for the complete learned model. The paper should distinguish properties proved for special constructions from properties supported only by experiments.
5- The learning experiments do not fully isolate the sources of improvement. The TRC–TRI comparison concerns fixed-parameter inference. A corresponding parameter-learning comparison, together with a matched test of the consistency mechanism, would help show whether the gains mainly come from region selection, the neural architecture, or the constraints. This is especially relevant because the neural baselines use different architectures and training settings.
Questions for authors
1- Could the parameter-learning results be evaluated on an independent test set and repeated across several data-generation and optimization seeds?
2- Does Proposition 1 apply to the exact parent-grouping construction used in Algorithm 1 and the experiments? What additional argument is needed for general sparse Bayesian networks?
3- How much of the large-data speed advantage remains when computing resources are matched and region-construction time is included? What is the peak memory requirement?
4- How does the method perform when more values are missing or variables have more than two states?
5- Could the authors add parameter-learning controls using TRC regions and the same neural architecture, and explain how the reference marginals for the high-treewidth inference benchmarks were obtained?
10 comments
Response to Weakness 2 and Question 4 — Higher missingness rates and non-binary variables
W2 & Q4: Report variability across repeated runs and independently generated data, and test higher missingness rates and non-binary variables.
Reply to Weakness 2 and Q4: Thank you for suggesting a broader evaluation. We agree and have added two more sets of experiments to test whether the parameter-learning advantage extends to settings with fewer observations or more variable states.
Higher missingness rates. We increased the missingness rate from 30% to 50% and 70% on the same 16 BNs. We ran each method independently three times per setting and evaluated it on independent test data. TRNK achieved the lowest average test NLL and CPD KL at both missingness rates (Table R2). Relative to the best baseline for each metric, TRNK reduced test NLL and CPD KL by 57.8% and 47.3%, respectively, at 50% missingness. At 70% missingness, the corresponding reductions were 24.1% and 44.5%. TRNK thus retained its average advantage in prediction and CPD recovery even when most values were missing.
Table R2. Parameter learning at 50% and 70% missingness (Fig. 9(a–b, d–e)). Best baselines: IJGP+DAMP-EM for test NLL and VMP for CPD KL.
| Missingness rate | Metric | TRNK | Best baseline | Reduction | BNs where TRNK ranks first |
|---|---|---|---|---|---|
| 50% | Test NLL | 7.35E-1 | 1.74E+0 | 57.8% | 11/16 |
| 50% | CPD KL | 1.20E-1 | 2.27E-1 | 47.3% | 13/16 |
| 70% | Test NLL | 3.16E+0 | 4.17E+0 | 24.1% | 8/16 |
| 70% | CPD KL | 1.78E-1 | 3.21E-1 | 44.5% | 10/16 |
Non-binary variables. We added a dynamic Bayesian network with three-state variables, a Grid network with four-state variables, and a Dense network with a mixture of two-, three-, and four-state variables. We kept the missingness rate at 30%. All five methods ran on three independently sampled training and validation splits with different optimization seeds, for a total of 45 fits. For each network, all methods shared the same independent test set of 10,000 records. TRNK achieved the lowest mean test NLL and CPD KL on all three networks (Table R3). Compared with the best baseline on each network, TRNK reduced mean CPD KL by 45.2%, 44.0%, and 28.6%, respectively. The original CPD recovery advantage thus extended to all three non-binary settings tested. Appendix E.5 describes the experimental setup.
Table R3. Three-run means for parameter learning with non-binary variables (Table 10).
| BN | CPD KL (TRNK) | CPD KL (baseline) | KL reduction | Test NLL (TRNK) | Test NLL (baseline) |
|---|---|---|---|---|---|
| HMM | 1.07E-1 | 1.96E-1 | 45.2% | 7.23E+0 | 7.40E+0 |
| Grid | 4.37E-2 | 7.79E-2 | 44.0% | 1.22E+1 | 1.23E+1 |
| Dense10 | 1.58E-1 | 2.22E-1 | 28.6% | 6.13E+0 | 6.27E+0 |
Revisions are highlighted in blue in the revised manuscript.
Response to Weakness 3 and Question 3 — Efficiency on large datasets
W3 & Q3: Account for GPU/CPU differences, omitted region-construction time, and VMP runs stopped before convergence. Report end-to-end runtime, peak memory, and results with matched resources.
Reply to Weakness 3 and Q3: Thank you for the feedback concerning the fairness of the comparison between VMP and TRNK. And for the missed end-to-end runtime, memory use, etc. We agree and therefore have added additional experiments.
For matched computing resources, we compared amortized VI (MF-VAE) on the same GPU platform and found that TRNK offers both accuracy and efficiency advantages. VMP, on the other hand, is not a GPU-native method, so it must run on a CPU with limited scalability; thus, the comparison is cross-platform. In contrast, TRNK is designed to fully use the advantages of GPU parallelization. Therefore, running TRNK on CPUs is somewhat compromised. The difference between the two algorithms' mechanisms is:
TRNK's amortized inference is designed to parallelize computation across samples. TRNK shares a single inference network across samples, allowing it to distribute the computation of region beliefs for a batch across multiple GPUs. The BayesPy VMP implementation used in our experiments updates each sample's variational state iteratively and runs on CPU. For these implementations, CPU is the common hardware platform. A single-core comparison measures update costs under a shared CPU budget. TRNK's multi-GPU implementation exercises the parallelism for which its large-sample training was designed. So, the original comparison measures practical training efficiency with each implementation running on its supported hardware.
To quantify the benefit of GPU parallelism, we held the code, data, and global batch size fixed on Grid5 with 200,000 samples and scaled TRNK from one to five A800 GPUs. The median time per training epoch decreased from 1.031 s to 0.227 s, a speedup, with virtually identical validation NLL. This shows that TRNK is a GPU-native method.
As shown in Table 6(C), on Grid5 with one million samples, the first two updates on the same CPU core took 42.79 s for VMP and 1764.76 s for TRNK. These CPU timings reflect the cost of the first two updates. Under the same single-core CPU budget, TRNK was slower than VMP over the first two updates. However, this setup limits TRNK’s ability to use parallelism across samples, which drives its efficiency on GPUs.
Including region construction preserves the original runtime comparison under the reported hardware configurations. Region construction accounts for only 0.10–0.74% of TRNK's total runtime across the nine settings. In the six HMM and Grid settings where VMP converged, TRNK was faster in five and achieved lower CPD KL in all six (Tables 6(A) and 1). On Grid5 with one million samples, TRNK took 372.90 s, compared with 522.63 s for VMP, and achieved approximately 91% lower CPD KL. For the three Promedas tests, the reported VMP checkpoints were taken at the two-hour time limit, before convergence. This is because we found the VMP iterations updated too slowly to converge within the time budget for these tests.
Table R4. Grid5 replay time and peak memory (Table 6(B)).
| Grid5 measurement | TRNK | VMP |
|---|---|---|
| 200k, end-to-end replay (s) | 116.42 | 77.41 |
| 200k, peak host PSS (GiB) | 9.40 | 0.62 |
| 200k, peak combined GPU memory (GiB) | 30.37 | N/A |
| 1M, peak host PSS (GiB) | 1.92 | |
| 1M, peak combined GPU memory (GiB) | N/A |
The 200k replay (Table R4) ran through the stopping criterion. The combined GPU memory peak is the maximum simultaneous sum across the five GPUs. The 1M TRNK memory peaks were measured over three training epochs.
Revisions are highlighted in blue in the revised manuscript.
Response to Weakness 1 and Question 1 — Independent testing and repeated experiments
Reply to weakness 1 and Q1: Thank you for suggesting an independent test set. We agree and therefore conducted supplementary experiments.
To address this concern, we kept the original fitted models fixed and independently generated 10,000 complete test records for each BN. All methods were evaluated on the same test set for each network. We used validation data to select hyperparameters and checkpoints and reserved the test data for final evaluation. TRNK retained the lowest average ΔNLL on the independent test sets. Compared with the best baseline VMP, TRNK's average ΔNLL was 42.8% lower on validation and 44.6% lower on the independent test sets (Table R1). The number of networks on which TRNK ranked first increased from 12 to 13.
We then tested whether this advantage would persist with new training samples and optimization seeds. We trained each method twice more, independently sampling the training and validation sets and generating fresh missingness masks for each run. The three runs used optimization seeds 42, 43, and 44, and we evaluated them on the same independent test set for each network. Giving the 16 networks equal weight, TRNK achieved a test ΔNLL of 0.12873±0.00077, compared with 0.22953±0.00795 for VMP, in nats per record. The values after ± are sample standard deviations across the three runs. TRNK's average test ΔNLL remained 43.9% lower than VMP's, and TRNK ranked first on 13 of the 16 networks by the three-run mean (Fig. 4(a)). The original comparative result thus held across different training samples, missingness masks, and optimization seeds.
Table R1. Validation and independent test ΔNLL (nats/case). Three-run results are from Table 4.
Revisions are highlighted in blue in the revised manuscript.