- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents BB-Net, a neural network trained with supervision for predicting the code distance of Bivariate Bicycle (BB) codes. The architecture includes a polynomial encoder to capture the local algebraic structure and a Tanner spectral encoder for global properties. Shift canonicalization and augmentation are introduced to enable effective training. The experiments show promising results in distance prediction within a tolerance, and comparable performance with heuristic methods for distance screening.
Critical strengths
- Distance prediction is a timely and important problem for fault-tolerant quantum computing, which the field is gradually transitioning into.
- The paper presents well-thought-out representation and architecture which captures the structure of the problem. The network runs with high efficiency compared with heuristic methods. The authors conducted rather extensive experiments, including heuristic label prediction, evaluation on certified distances, distance screening analysis, and ablation studies.
Critical weaknesses
The main concerns lie in the experiments. (1) The headline “96.39% accuracy” allows an error of , which is somewhat misleading. Moreover, it is measured against heuristic labels instead of certified true distance. While this effectively evaluates the accuracy of the learning task and is understandable for large instances, it isn't persuasive enough for accuracy for true distance prediction. (2) The performance on the certified subset and distance screening are less favorable compared with strongest heuristic methods. This downstream application however, in my understanding, is the primary goal of distance prediction. Also, the screening evaluation is limited to AUC. More evidence of its practical advantage would help strengthen the paper.
Questions for authors
- What is the exact accuracy without tolerance on the test benchmark?
- Does the distance screening uses comparable compute budget? BB-Net has a good performance-speed tradeoff in my opinion, which isn't sufficiently shown in this result.
- I want a clear description of the distribution of (l, m) in training and test dataset. A discussion about the generalization of BB-Net over (l, m) would be meaningful.
6 comments
Clarifications and Additional Experiments [2/2]
3. Following the reviewer request we add l,m distribution for both train and test set. The training and test codes consist of pairs of l and m from 3 to 30 under the constraint of n=2lm<=360.
Train:
| l \ m | 3–9 | 10–16 | 17–23 | 24–30 | Sum |
|---|---|---|---|---|---|
| 3–9 | 7.28% | 4.49% | 7.79% | 7.01% | 26.56% |
| 10–16 | 23.13% | 9.95% | 0.21% | . | 33.29% |
| 17–23 | 22.03% | 0.87% | . | . | 22.90% |
| 24–30 | 17.25% | . | . | . | 17.25% |
| Sum | 69.68% | 15.31% | 8.00% | 7.01% | 100% |
Test:
| l \ m | 3–9 | 10–16 | 17–23 | 24–30 | Sum |
|---|---|---|---|---|---|
| 3–9 | 12.56% | 1.77% | 3.58% | 2.93% | 20.83% |
| 10–16 | 28.25% | 8.43% | 0.05% | . | 36.73% |
| 17–23 | 22.86% | 1.06% | . | . | 23.92% |
| 24–30 | 18.52% | . | . | . | 18.52% |
| Sum | 82.18% | 11.26% | 3.63% | 2.93% | 100% |
Following the reviewer request, we tested BBNet on 32 out of distribution l,m lattice sizes codes and got a good result. BBNet exact match has dropped significantly, however, it has managed to maintain close predictions to the distance achieving 93.8% on accuracy with tolerance of 2.
| Acc Exact ↑ | Acc@2 ↑ | MAE ↓ | RMSE ↓ | |
|---|---|---|---|---|
| BBNet | 59.4% | 93.8% | 0.938 | 1.541 |
The constraint of n=2lm<=360 is preserved. Example of l,m codes that were tested are:
| n | k | l | m | Distance |
|---|---|---|---|---|
| 276 | 8 | 46 | 3 | 8 |
| 360 | 8 | 3 | 60 | 18 |
| 252 | 4 | 42 | 3 | 4 |
| 324 | 4 | 54 | 3 | 18 |
| 306 | 4 | 3 | 51 | 22 |
[1] - Cruz-Benito, Juan, et al. "Evolutionary Discovery of Bivariate Bicycle Codes with LLM-Guided Search." arXiv preprint arXiv:2606.02418 (2026).
Thank you for your response and the additional experiments. The exact match and distance screeing experiments effectively addressed my first and second concern.
Some followups regarding the remaining points: (1) In Response 2, FOM is provided as upper bound. Were the candidates independently evaluated with a heuristic or exact solver? (2) The distribution of (l,m) seems quite uneven. Could the authors clarify the reason?
Clarifications and Additional Experiments [1/2]
We thank the reviewer for their careful reading of our manuscript and for their constructive feedback. We address each concern individually, providing additional explanations, experimental evidence, and clarifications where appropriate. We will incorporate the relevant clarifications and additional results into the revised manuscript.
1. Distance prediction proximity to the ground truth distance has a meaning. For example, prediction of 14 to a ground truth distance of 15 is a lot better than prediction of 22. Thus, the task at hand is not a standard categorical classification problem. RMSE, MAE are better metrics than accuracy to measure performance. However, accuracy is a more intuitive metric than the above. We introduce the accuracy tolerance to present accuracy while still consider the proximity factor. Following the reviewer request we also add the exact match accuracy results:
2. The reviewer correctly identifies the heuristics in distance screening have larger compute budget than BB-Net, hence, are not comparable. We have ran this experiment again but with less compute for the heuristics, reduced to 10 trials while stim based methods remained the same (1 trial and 3 for UEstim) approaching BBNet compute, however, still a lot less efficient. These are the results:
Following the reviewer request, we conduct an additional experiment to better emphasize the importance and the good performance-speed tradeoff of BBNet.
We use IBM's pipeline for searching optimal BB codes with an LLM [1]. This research presents a workflow where an LLM generate code candidate, then BP-OSD evaluate the distance and only high upper bound distance continue to the expensive part of exact distance computation. We present a short experiment demonstrating a more extensive search using BBNet instead of BP-OSD, finding better candidates with the same time budget. The experiment results are:
During this search, BB-Net managed to evaluate nearly twice as many codes as BP-OSD, finding better candidates with better FOM. The top three candidates found by each search are as follows:
BP-OSD
BB-Net
This experiment demonstrates the effectiveness and the importance of BBNet in a real code search pipeline.