- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper formulates finite-(K) posterior learning by minimizing the probability that all (K) independently sampled solvers fail. It derives an exact Fenchel reformulation and uses Bregman/EMA tracking of problem-specific weights, allowing IVON posterior updates with one parameter draw. Experiments on recursive reasoning and Qwen3-4B report improved Pass@(K) and substantially lower per-update cost than Direct-(K).
Critical strengths
- The finite-(K) objective directly matches candidate coverage, and the Fenchel derivation clearly exposes the problem-specific weighting.
- The method is evaluated on recursive and autoregressive models. The matched comparison with Direct-10 shows comparable coverage with substantially lower measured update time and FLOPs.
Critical weaknesses
-
Several central differences are small, for example, the average language-model gain over DMPO is 0.84 points, and the Qwen experiment uses one training seed. Error bars and matched multi-seed comparisons are needed to determine whether these gains are robust.
-
The gains in the paper are not clearly tied to finite-K weighting. On MATH, λ = 1 on the first visit and each problem is seen about 1.28 times. On Sudoku, the EMA barely moves from its initial value. On Maze, K_train = 1 already reaches 90.8 Pass@10. Maybe a constant-weight IVON baseline is needed?
-
Most margins are within seed noise. The same Maze run gives 91.7 for one seed and 91.3 averaged over three seeds. Key comparisons differ by only 0.1–0.6 points. The LLM result is from a single seed, and the average lead comes mostly from AIME25 with only 30 problems.
Questions for authors
See the weakness above.
7 comments
Response 1: Error Bars and Matched Multi-Seed Comparisons
W1) The small performance differences, including the 0.84-point average LLM gain over DMPO from one training seed, need error bars and matched multi-seed comparisons to establish robustness.
Thank you for pointing out that the missing error bars and single-seed LLM comparison made the gains difficult to assess. The repeated LLM results retain FBI's average Pass@8 advantage, and we now report the spread of the recursive benchmark results.
Repeated results are mean ± SE across three independent training runs per method. P-values use two-sided paired t-tests on matched runs. At , O denotes a significant FBI advantage and X a significant comparator advantage; △ denotes no significant difference.
1. Matched LLM comparisons across training seeds
We repeated training for FBI and all five trained baselines in the autoregressive LM setting with the same five-benchmark evaluation protocol. Each benchmark column reports Pass@8 (%), and Avg. is the unweighted mean across the five benchmarks, computed within each run before aggregation.
| Method | MATH500 | OlymMATH | AMC23 | AIME24 | AIME25 | Avg. | p vs. FBI (Avg.) |
|---|---|---|---|---|---|---|---|
| Base | 87.80 | 53.57 | 75.18 | 34.15 | 21.56 | 54.45 | N/A |
| GRPO | 89.44 ± 0.55 | 60.72 ± 0.72 | 79.38 ± 0.47 | 28.37 ± 0.36 | 26.12 ± 1.55 | 56.81 ± 0.16 | <0.001 (O) |
| Pass@ Training | 91.30 ± 0.35 | 61.91 ± 0.99 | 83.49 ± 0.52 | 32.34 ± 0.54 | 26.25 ± 2.04 | 59.06 ± 0.43 | 0.045 (O) |
| DMPO | 90.40 ± 0.51 | 61.45 ± 0.20 | 81.43 ± 1.17 | 31.54 ± 0.98 | 27.61 ± 0.92 | 58.48 ± 0.55 | 0.042 (O) |
| MaxRL | 91.08 ± 0.29 | 62.88 ± 0.82 | 83.79 ± 1.13 | 30.25 ± 0.88 | 26.24 ± 1.37 | 58.85 ± 0.47 | 0.067 (△) |
| MMPO | 90.78 ± 0.43 | 60.80 ± 0.96 | 78.98 ± 1.51 | 28.57 ± 1.54 | 25.32 ± 1.22 | 56.89 ± 0.68 | 0.026 (O) |
| FBI | 91.48 ± 0.08 | 60.45 ± 0.30 | 84.46 ± 0.42 | 31.19 ± 0.24 | 33.46 ± 0.12 | 60.20 ± 0.18 | — |
The submitted 0.84-point lead over DMPO was from a single training seed. Across the repeated runs, the average lead is 1.72 percentage points over DMPO () and 1.14 points over Pass@ Training (), the strongest baseline by repeated-run average. FBI's mean exceeds MaxRL's by 1.35 percentage points, but the paired comparison is not significant ().
The advantage is aggregate: MaxRL leads on OlymMATH, while Pass@ Training and DMPO remain higher on AIME24. Table 2 of the revised manuscript now includes the full repeated-run results. For AIME25's contribution to the overall average, see Response 3.
2. Recursive benchmark comparisons with uncertainty estimates
The following table summarizes candidate coverage using the TRM backbone. All performance entries are percentages.
| Benchmark | Metric | PTRM | W-PTRM | FBI | p vs. PTRM | p vs. W-PTRM |
|---|---|---|---|---|---|---|
| Sudoku-Extreme | Pass@10 | 97.38 ± 0.25 | 98.84 ± 0.29 | 99.55 ± 0.25 | <0.001 (O) | 0.004 (O) |
| Maze-Hard | Pass@10 | 88.40 ± 0.16 | 89.80 ± 0.34 | 91.30 ± 0.21 | 0.002 (O) | 0.009 (O) |
| ARC-AGI-1 | Pass@100 | 60.25 ± 1.06 | 61.25 ± 1.16 | 64.25 ± 0.90 | 0.005 (O) | 0.011 (O) |
| ARC-AGI-2 | Pass@100 | 11.94 ± 0.66 | 10.69 ± 1.06 | 12.78 ± 0.72 | 0.016 (O) | 0.027 (O) |
FBI's coverage improvements over PTRM and W-PTRM have on all four benchmarks. With the FPRM backbone on Maze-Hard, FBI also improves Pass@10 from W-FPRM's 88.70 ± 0.24% to 94.80 ± 0.62% (). The revised manuscript reports the recursive results with SEs in Table 1 and the additional FPRM comparison in Figure 3(a).
Response 2: Contribution of Finite- Weighting beyond IVON and Simpler Controls (1/2)
W2) The contribution of finite- weighting is unclear: MATH starts with and averages only about 1.28 visits per problem, Sudoku's EMA stays near its initial value, and already reaches 90.8% Pass@10 on Maze-Hard. A constant-weight IVON baseline is needed to isolate the weighting contribution.
Thank you for highlighting the need to separate posterior learning from the contribution of finite- weighting. Matched controls show that finite- weighting adds coverage gains beyond constant-weight IVON on Maze-Hard and the autoregressive LM, including under the latter's limited-revisit schedule. On Sudoku-Extreme, updating the EMA also improves coverage over freezing it at initialization, directly addressing the near-initial-estimate concern.
Within each weighting ablation, we keep IVON, initialization, loss processing, KL regularization, and the training budget fixed. We compare the following weighting rules:
- Constant weights: , retaining posterior learning without problem-specific weights.
- Minibatch-mean weights: one shared weight per minibatch, preserving its mean FBI weight while removing differences between problems.
- OHEM-style EMA hard mining [1]: retain the hardest 50% of problems by their EMA failure estimates, including cutoff ties.
- Rank-based EMA soft weighting [2]: use positive exponential weights based on EMA difficulty ranks, with selection pressure 100.
Hard mining and soft weighting adapt the cited methods to EMA-based loss weighting and match the mean FBI weight computed from each control's own estimates. These controls distinguish the finite- weight mapping from posterior learning, a shared weight scale, and simpler uses of the same difficulty estimates. P-values use two-sided paired t-tests over three matched training runs. At , O denotes a significant FBI advantage and X a significant comparator advantage; △ denotes no significant difference.
1. Maze-Hard: gains beyond a strong posterior learner
The weighting ablation in Appendix C.1, Table 16 retains and one posterior draw per update.
| Weighting | Pass@10 (mean ± SE, %) | p vs. FBI |
|---|---|---|
| Constant-weight () | 90.60 ± 0.22 | 0.003 (O) |
| Minibatch-mean weights | 90.70 ± 0.19 | 0.018 (O) |
| OHEM-style EMA hard mining | 90.90 ± 0.17 | 0.037 (O) |
| Rank-based EMA soft weighting | 90.90 ± 0.14 | 0.046 (O) |
| FBI | 91.30 ± 0.21 | — |
FBI improves Pass@10 by 0.70 percentage points over constant-weight IVON and 0.60 points over minibatch-mean weights. It also exceeds both hard mining and soft weighting by 0.40 points, with and , respectively. Together, these comparisons support a contribution from the finite- problem-specific mapping beyond a shared weight scale or difficulty ranking alone (Appendix C.1, Table 16).
The control has , so its Pass@10 result is also 90.60 ± 0.22%. We use this same control result and paired comparison () in the training-target and weighting ablations (Appendix C.1, Tables 15–16).
2. Autoregressive LM: contribution under limited revisits
At , the weight equals one, so constant-weight IVON also serves as the control in this ablation. All conditions retain eight responses per prompt, and FBI uses . The controls use the same short training schedule, so the limited opportunity for revisits applies to all conditions.
| Weighting | Avg. Pass@8 (mean ± SE, %) | p vs. FBI |
|---|---|---|
| Constant-weight IVON () | 59.08 ± 0.27 | 0.015 (O) |
| Minibatch-mean weights | 58.96 ± 0.29 | 0.009 (O) |
| OHEM-style EMA hard mining | 59.39 ± 0.47 | 0.113 (△) |
| FBI () | 60.20 ± 0.18 | — |
Avg. is the unweighted mean across the five mathematical-reasoning benchmarks. FBI improves average Pass@8 by 1.12 percentage points over constant-weight IVON () and 1.24 points over minibatch-mean weights (). Its 0.81-point average advantage over hard mining is not statistically significant (; Appendix C.1, Table 17).
At the same limited-revisit budget, the significant gains over constant and minibatch-mean weights support a measurable contribution from problem-specific reweighting. Thus, infrequent revisits do not make FBI equivalent to constant-weight IVON in the evaluated autoregressive LM setting.
Summary of Responses
Thank you for recognizing the finite-K objective's direct connection to candidate coverage and the efficiency of the matched Direct-10 comparison. Your comments helped us strengthen the evidence on training variability, isolate the contribution of finite-K weighting, and clarify the scope of the LLM gains. Below, we summarize these additions and their main findings. We will provide detailed answers to each concern in separate responses (Responses 1–3), including the supporting results and theoretical analysis.
1. Repeated training and paired tests support the reported gains.
2. Matched controls support a contribution beyond IVON alone.
3. AIME25 drives the largest gain, with improvements also on MATH500 and AMC23.