acceptodds
Under review as a conference paper at ICLR 2027

Using Probabilistic Programs to Train Inductive Reasoning in Large Language Models

Abstract

Post-training Large Language Models (LLMs) for reasoning typically focuses on deductive tasks such as mathematics and coding where correctness is verifiable. Yet, many real-world reasoning problems are inductive: agents must infer uncertain beliefs from sparse, ambiguous observations. For standard fine-tuning methods, it is often challenging to curate large-scale, high-quality labeled datasets and to handle targets that are inherently distributional. In this work, we introduce a novel approach, called Program-based Posterior Training (PPT), to address these limitations: we use an LLM to generate diverse open-world scenarios as probabilistic programs, run probabilistic inference to produce distributional target responses to queries, and then fine-tune on these probabilistic labels. Using this approach, we fine-tune LLMs on 10,000 programmatically generated scenarios and evaluate on held-out motifs, human-labeled judgments, and external benchmarks. Overall, PPT substantially improves estimation accuracy on held-out inductive tasks, increases alignment with human judgments, and transfers to external benchmarks for estimation and calibration. Additionally, the gains in raw calibration are not subsumed by post-hoc temperature scaling, showing that the models have more deeply internalized uncertainty compared to output rescaling. Together, these results suggest that probabilistic-program-mediated fine-tuning is a promising approach for post-training LLMs to perform stronger approximate inductive inference.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large language models, Bayesian inference, inductive reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Ca9fOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper addresses post-training LLMs for inductive reasoning, i.e., inferring uncertain latent quantities and predictions from sparse natural-language observations, a setting where labeled data are scarce and targets are inherently distributional. It proposes Program-based Posterior Training (PPT), which extends the Model Synthesis Architecture (MSA): an LLM generates natural-language scenarios with queries in sports, healthcare, and general domains, translates each scenario into a Pyro probabilistic program, and posterior inference (MCMC or rejection sampling) yields a posterior over each query's answer on a 0–100 scale. The discretized posterior serves as a soft target in a cross-entropy loss over integer answer tokens, used to LoRA-fine-tune Llama3-8B-Instruct and Qwen2-7B-Instruct, with additional Qwen2.5-14B/72B runs. The dataset contains roughly 15k scenarios and 60k queries, split by inferential motifs. Evaluation covers mean absolute error against program posteriors on held-out queries, motifs, and domains; comparisons with Gemini-3.1-Pro and a model fine-tuned on Gemini answers; correlation with previously published human judgments; and accuracy, NLL, and ECE on OpenEstimate, PosteriorDB, Bayesian Teaching, and five multiple-choice benchmarks, alongside temperature scaling, chain-of-thought with self-consistency, mean-target, and forward-sampling variants.

Critical strengths
  1. A practical source of distributional supervision.
    PPT turns probabilistic program synthesis into a training-data pipeline for tasks where uncertainty matters and human labels are costly. The approach is conceptually simple and potentially reusable.

  2. Consistent empirical improvements.
    Synthetic sports MAE decreases from approximately 10.8 to 5.0–5.7 (Fig. 2), and human alignment improves from R2=0.509R^2=0.509 to 0.7750.775 (Fig. 3). Raw NLL and ECE improve across the reported evaluations for both primary backbones, with benefits generally remaining after temperature scaling (Table 2).

Critical weaknesses
  1. The benefit of posterior-specific structure remains unclear.
    Although PPT generally complements temperature scaling, the experiments do not isolate posterior-specific supervision from broader soft-label effects. The mean-target comparison changes both uncertainty and distributional shape, while forward sampling also changes the supervision procedure and potentially data volume. Forward sampling achieves lower raw ECE in five of seven columns of Table 4. The results therefore support improved uncertainty estimates more directly than the stronger claim of “more deeply internalized uncertainty.”

  2. Generalization and label-quality claims require qualification.
    The displayed synthetic domains largely share one broad inferential structure, limiting what motif and domain holdouts establish. PosteriorDB provides relevant external evidence, but its evaluation construction needs clarification. Furthermore, agreement with program posteriors does not independently establish that program-derived labels better represent the intended scenarios than Gemini-derived labels. The MSA, ForestDB, and human-alignment checks are useful, but the PDF does not report a representative semantic audit of the generated corpus.

Questions for authors

Data generation and experimental setup

  1. Data construction and splits.
    Which models generated the scenarios and programs? Please report filtering rates, semantic validation procedures, exact answer-scale mappings, and the splits used for tuning and early stopping.

  2. Evaluation protocol.
    Please specify external-task prompts, probability normalization and NLL definitions, the temperature-fitting split, and seed counts and uncertainty statistics.

Attribution of the gains

  1. Posterior structure.
    Under matched training conditions, how does PPT compare with simplified soft targets that preserve posterior means while controlling entropy or variance? Distribution-sensitive metrics would complement MAE and ECE.

  2. Label source.
    What target format does Gemini-Answer-FT use? A comparison with matched label formats, including evaluation on the human judgments in Fig. 3, would better isolate label-source quality.

Additional Suggestions

  1. Generalization.
    Report the PosteriorDB model families, their natural-language conversion, and per-family results to clarify structural transfer.

  2. Backbone coverage.
    The Qwen2.5-14B/72B results provide useful additional coverage; a more recent backbone would be an optional extension.

  3. Scalability.
    Report data-generation cost, inference compute, and usable-example yield.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Ca9f

We thank the reviewer for helpful feedback, which will improve the paper.

General Questions

Q: The benefit of posterior-specific structure is unclear.

A: We view forward sampling and posterior sampling as two versions of our pipeline, and we view the pipeline and the data created with labels from probabilistic programs as the source of improvement. The two versions arise naturally from the idea of scenario + program generation. We will bring extra care and adjust any claims that suggest that distributional labels are the performance driver or a key contribution. Forward sampling’s advantage lies in having more data: it can be thought of as training on individual noisy samples behind posterior sampling, whereas posterior sampling trains on that one posterior distribution estimated from many samples. Table 4 shows that they achieve similar performance, reflecting that the noisier data from forward sampling is partly offset by its large scale.

The key result is that training language models with data generated from probabilistic programs results in significant improvements in their probabilistic reasoning ability and calibration, with different data generating methods (forward sampling and posterior sampling) being effective in different settings.

Q: Generalization and label-quality.

A: We agree that the synthetic domains share one broad family of inference: latent traits inferred from noisy outcomes written in a broadly similar template. Therefore, for empirical evaluations we test trained models in a progression of similar data to external data: held-out motifs from our synthetic data, and completely external datasets like OpenEstimate, PosteriorDB and the multiple-choice benchmarks.

Program vs. Gemini labels. While the paper extends beyond agreement with program posteriors with human judgement, it is only comparing base and PPT model with human judgements. For the below table we ran this comparison on more models, including the Gemini-distilled one. At matched format (one-hot on a single integer, rather than on a distribution), the model trained on program means agrees better with the human judgments of Fig. 3 than the model trained on Gemini's answers.

Base modelPPT (Dist)PPT (Mean)Gemini-Answer-FT
R² (mean of the 9 panel R² values).509.769 ± .003.819 ± .004.735 ± .007
MAE to human mean13.117.62 ± 0.166.31 ± 0.0710.24 ± 0.14

Semantic audit. Every example in the corpus passes automated checks: at least 1000 accepted rejection-sampling draws, and a check that each query returns the quantity its answer convention asks for; otherwise the example is rejected. An LLM judge is not a reliable substitute at scale, because strong LLMs answer these queries poorly (Fig. 2). Downstream evaluation is itself a check on label quality: semantically wrong labels would not improve agreement with human judgments and external benchmarks. We additionally ran a controlled ablation with the same training but with labels and scenarios shuffled. It loses those gains by having only a 0.2 correlation with humans, much lower than the base model’s 0.509 shown above.

PosteriorDB construction. 72 scalar queries on six posteriors (eight schools, surgical, seeds, dugongs, kilpisjarvi,mesquite): 45 queries on posterior means and 27 queries on posterior probabilities of an event (e.g., P(theta_A > theta_C)). The prompt states the Bayesian model in prose (likelihood and priors), lists the observed data, and asks one query on a stated 0-100 grid, in the same answer format. Gold answers come from PosteriorDB's own published programs.

Q: Attribution of the gain. Posterior structure. Under matched training conditions, how does PPT compare with simplified soft targets that preserve posterior means while controlling entropy or variance? Distribution-sensitive metrics would complement MAE and ECE.

A: Our matched control is PPT (Mean) (Table 2). It uses the same questions, recipe, hyperparameters, seeds and selection rule as PPT (Distribution), but its target is a point mass at the rounded posterior mean. It matches PPT on point accuracy (held-out-scenario MAE 5.1 vs. 5.6; OpenEstimate 21.8 vs. 22.9) but loses on every distribution-sensitive metric.

AuthorsOpenReview ↗

Continued Response

Implementational Detail Questions

Q: Data generation and experimental setup. Data construction and splits. Which models generated the scenarios and programs? Please report filtering rates, semantic validation procedures, exact answer-scale mappings, and the splits used for tuning and early stopping.

Evaluation protocol. Please specify external-task prompts, probability normalization and NLL definitions, the temperature-fitting split, and seed counts and uncertainty statistics.

A: Data. Gemini-3.1-Pro writes the scenarios (temperature 0.5) and the Pyro programs (temperature 0, MSA's three prompting steps). No LLM labels any answer: labels come from exact rejection sampling in each program (1,000 accepted draws). Of 19,296 scenarios, 85.4% yield a working program; most losses are sampler timeouts where 1000 accepted draws are not reached. We drop a further 1.8% of examples with unreliable posteriors (>5% of draws outside 0-100, or 0/1 outcomes instead of probabilities). At the pipeline level, our programs agree with ForestDB reference programs (posterior-mean MAE 0.017) and with human judgments (r = .71). This validation is summarized at the end of section 3.1, and we will add the other details in Appendix A.2.

Answer scale. Every answer is an integer 0-100: probabilities x 100. The distributional target is the histogram of the rounded posterior draws over the 101 integer tokens. This is defined in main text section 3.2, and how the prompt communicates this to LLM when answering the question is shown in appendix section A.2.

Splits and selection. We split train-validation-test by motif: in every domain, two of the 16 motif combinations are held out and divided 50/50 into validation and test (56,481 / 4,180 / 4,195 examples). All runs use one setting (LoRA r = 8, lr 5e-5, 500 epochs), and we keep the checkpoint with the lowest loss on a fixed 773-example validation subset. There is no early stopping, and the test split is never used for selection. Motifs are discussed in Section 3.3, and we will further clarify train/val/test in Section 4.

Evaluation. Appendix A.2 lists most of our prompts, but here we add how the answer is scored: All tasks are zero-shot, scored from one teacher-forced pass. Multiple-choice prompts give the question and lettered choices, and the assistant turn is prefilled with "The answer is: <". Section 3.2 includes relevant definitions, and to further add to that, our evaluation NLL = -log p(gold), with p a softmax over the letters A-Z for multiple choice questions, or over integers from 0 to 101 for numerical questions (always converted to 0 to 100 scale and instructed to output integers). Temperature scaling fits one temperature per model and benchmark on up to 1,000 items from the benchmark's training split. Each fine-tuned model is trained with 3 seeds, and tables report the mean +/- standard error (SD / sqrt 3). These details will be added to Appendix A.2.

Q: Scalability. Report data-generation cost, inference compute, and usable-example yield.

A: Data-generation cost: about 12k input + 29k output tokens per program. Inference compute: exact rejection sampling on 1 CPU core per scenario, median 12 minutes; about 8,200 CPU-hours in total, no GPU. Fine-tuning: one 40 GB A 100 GPU, 14-24h per run. At test time the model answers in one forward pass. Usable-example yield: 85.4% of scenarios give a working program, and how-to-filter is answered above.

Thank you for the additional experiments and clarifications. While they strengthen the empirical results, concerns regarding the contribution of posterior-specific supervision and the semantic validity of generated programs remain. The generalization claims also require further qualification. Therefore, I maintain my original rating (2: Weak Reject).

Reviewer_bLm4OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies probabilistic inductive reasoning in LLMs from sparse natural-language observations. The authors propose Program-based Posterior Training (PPT), where an LLM generates scenarios and corresponding probabilistic programs, then posterior inference is run to obtain answer distributions for fine-tuning the LLM. Experiments show that LLM finetuned in this way improved agreement with program-derived targets and human judgments, transfer from sports to healthcare scenarios, and improved estimation and calibration on external benchmarks.

Critical strengths
  1. The proposed method is simple and effective.
  2. The paper provides extensive experiments on the effectiveness of the proposed method. The evaluation covers held-out scenarios and motifs, and human judgments. The results show improved posterior estimates and agreement with human judgments, alongside calibration gains across model scales.
  3. The model also improves outside the training domain. Sports-only training improves performance on healthcare scenarios, and the external benchmarks show gains in estimation and calibration. These results suggest that the model learns useful probabilistic thinking beyond matching the synthetic training targets, although accuracy gains are not consistent across all settings.
Critical weaknesses
  1. Direct program generation baseline. Since the training targets come from LLM-synthesized probabilistic programs, a natural baseline is to ask an LLM to generate a program for each test scenario and execute it to answer the questions. Although it might induce much higher inference cost, this experiment can shed light on how it compares with directly supervising over the distribution.

  2. Limited novelty. Distributional supervision is a known technique, and the program-synthesis procedure builds on previous work such as MSA. The contribution is mainly using program-induced posteriors as training targets and showing that this supervision transfers. This is useful, but the advance over prior work is limited.

Questions for authors

Why does forward-sampled supervision achieve substantially lower raw calibration error than posterior-distribution supervision on MMLU and TruthfulQA in Table 4?

2 comments
AuthorsOpenReview ↗

Response to Reviewer bLm4

We thank the reviewer for helpful feedback, which will improve the paper.

Q: Direct program generation baseline.

A: This is an insightful point. We agree that directly running the programs is interesting if the goal is to answer these kinds of queries well. Indeed for Figure 3, we find that fine-tuned LLM and programs have similar R^2 correlations with humans, although the figure itself only plots out the base and fine-tuned LLMs against humans. Aside from eventual inference-cost savings compared with running programs, fine-tuning LLMs on this probabilistic reasoning data also yields capabilities such as calibration that can externally transfer.

Q: Limited novelty. Distributional supervision is a known technique, and the program-synthesis procedure builds on previous work such as MSA. The contribution is mainly using program-induced posteriors as training targets and showing that this supervision transfers. This is useful, but the advance over prior work is limited.

A: We agree that it is not the first attempt at distributional supervision, and the sequential prompting procedure for program generation is adapted from MSA.

Data from the pipeline rather than strictly distributional labels as source contribution. We view a core contribution as developing a pipeline: programmatically generating scenarios, generating programs, running programs, getting labels, and fine-tuning. An insight this pipeline leverages is that directly answering the query can be challenging (sometimes to both LLMs and humans) as it asks for latent structures behind sparse observations. However, it is easier to construct a reasonable generative structure from possible latents to observations, instantiated by programs. The program-runs and sampling processes then do the heavy-lifting of getting labels in a scalable way.

Compared with MSA. There’s a pipeline difference and result difference. Pipeline is distinct from MSA by generating new scenarios with sequential prompting (whereas MSA focused on a few human generated scenarios), using Pyro instead of WebPPL for program generation (more robust; modern LLMs are better at Pyro), and fine-tuning LLMs on this data. Result-wise, given a fresh problem roughly in this domain, MSA still needs a freshly created probabilistic program and run it, whereas our fine-tuned LLM can directly answer this problem, saving both inference tokens and the need to run a probabilistic program.

Q: Why does forward-sampled supervision achieve substantially lower raw calibration error than posterior-distribution supervision on MMLU and TruthfulQA in Table 4?

A: We view forward sampling and posterior sampling as two versions of our pipeline, and the pipeline as the main contribution. These two versions arise naturally from the idea of scenario + program generation. We will bring extra care and adjust any claims that suggest that distributional labels are the performance driver or a key contribution. Forward sampling’s advantage lies in having more data: it can be thought of as training on individual noisy samples behind posterior sampling, whereas posterior sampling trains on that one posterior distribution estimated from many samples. Table 4 shows that they achieve similar performance, reflecting that the noisier data from forward sampling is partly offset by its large scale.

Area_Chair_Nz71OpenReview ↗

Engaging with the authors

Dear reviewer,

the authors have posted a reply to your review. Please have a look and interact with the authors as needed.

Thank you, Your AC

Reviewer_Sid5OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose Program-based Posterior Training (PPT). An LLM writes short open-world scenarios, many of them team competitions with latent skills and per episode factors. These are translated into Pyro programs, and posterior inference gives distributions over the queried quantities on a 0-100 scale. The authors LoRA finetune Llama3-8B and Qwen2-7B on these distributions. They report lower MAE against the program posterior means on held out motifs and domains, lower NLL and MAE on OpenEstimate and posteriordb, and lower NLL and ECE on Bayesian Teaching and five multiple-choice benchmarks. The improvements hold with temperature scaling and with chain-of-thought plus self-consistency.

Critical strengths
  • The approach sits between two existing ideas: writing a probability program for each question at test time, and training models on Bayesian teachers. It offers a scalable source of distributional training data.

  • The evaluation is broad and includes human data. Two model families, a scaling study, and agreement with human judgments, where fine-tuning raises R² from 0.509 to 0.775.

Critical weaknesses
  • The test set evaluation is very similar to the training set. Test scenarios come from the same pipeline, and the "correct" answers come from the same kind of AI-written programs. The model may simply be learning to imitate the program generator rather than learning to reason correctly. The program posterior means also fall mainly between 40 and 60 (Figure 1 inset), so a model that hedges toward the middle would score well on the MAE in Figure 2. The paper needs simple baselines to show the model is learning what the paper claims. What MAE does a model that always answers 50 get? What about always giving the typical answer for the question type, or a model fine-tuned on answers shuffled across scenarios?

  • Some of the scores appear to be about the same as pure guessing. If OpenEstimate and posteriordb are scored over the 101 values from 0 to 100, a model saying "every number is equally likely" would score NLL ~4.62. PPT scores 4.53-4.54 with Llama and 4.88 with Qwen on posteriordb (Table 2a), about the same as or worse than this. Only Qwen on OpenEstimate with 3.86 is better than guessing. The paper does not explain how these benchmarks are scored. Please report this pure-guessing reference. Looking at Table 4, a variant trained on single sampled answers (forward sampling) is better calibrated than the full-distribution version on 5 of 7 benchmarks (e.g., MMLU ECE 0.043 vs 0.112), which undercuts the claim that training on full distributions is what matters.

Questions for authors
  • Were any MCMC diagnostics run?
  • What MAE do a model that always answers 50, the typical answer for each question type, and a model fine-tuned on answers shuffled across scenarios get in Figure 2?
  • How are OpenEstimate and posteriordb posed and scored? For the multiple-choice tasks, are probabilities normalized over the answer options, and on which data was the temperature fitted? After temperature scaling, Winogrande NLL (0.80-1.03) is worse then a coin flip of 0.69.
  • Do the NLL and ECE gains survive a control that makes the model's outputs equally spread out without any program knowledge ( like label smoothing, or posteriors shuffled across questions)?
  • Which LLM generated the scenarios and programs, what fraction of programs failed or were filtered out, and is there a test split separate from the one used for tuning and early stopping?
4 comments
AuthorsOpenReview ↗

Response to Reviewer Sid5

We thanks the reviewer for helpful feedback, which will improve the paper.

Results interpretation / ablation related questions

Q: Test set is similar to train set, and what is the model learning? What MAE do a model that always answers 50, the typical answer for each question type, and a model fine-tuned on answers shuffled across scenarios get in Figure 2?

A: We agree that an ablation control is important to run. We ran a shuffled ablation that trains on PPT data with scenarios and posterior answer labels shuffled, so the scenario text gives no information to the labels, and the model would only learn the overall label distribution. We also tested always answering 50. On Figure 2 sports and healthcare validation (the second and third group of bar charts), the shuffled control gets MAE 11.18, 8.42, respectively; 50-constant gets 11.23, 8.34; both higher than PPT models in the 5 to 7 range. Furthermore, on the Figure 3 data with human answers, base-model, fine-tuned model, and shuffled control gets correlation r^2 of 0.51, 0.78, 0.2, respectively. The shuffled control also has a few percentage point lower accuracy on the external MCQ datasets. We will report this control baseline in the relevant paper sections.

In Section 4.1, we indeed report on held-out data with similar templates as the training set (we clarify validation vs. test in a different question). Meanwhile, we hold out motifs (motifs are “inference structure”, mentioned in Section 3.3, defined at the end of the appendix), hold out domains (healthcare vs. sports in Figure 2), and test human alignment to understand the model’s ability to reason with this style of probabilistic reasoning data.

Q: Do the NLL and ECE gains survive a control that makes the model's outputs equally spread out without any program knowledge ( like label smoothing, or posteriors shuffled across questions)?

A: We agree that part of any NLL/ECE improvement over an overconfident base model can come from simply spreading probability mass, so the relevant comparison is between models that have both been given the same spreading correction.

Temperature scaling (already in the paper). Our main calibration table includes a post-hoc temperature-scaling (TS) block for both the base model and PPT, with temperature T fit on held-out training-split examples. After both models are temperature-scaled, PPT still has lower NLL than the base model on every Llama-3 benchmark and on every Qwen-2 benchmark except HellaSwag.

Label smoothing (new). Following the reviewer's suggestion, we added a post-hoc label-smoothing (LS) control. Each model's answer distribution is mixed with a uniform distribution over the question's answer options, p' = (1 - eps) p + eps / k, with one eps per benchmark and model, chosen to minimize NLL. For the multiple-choice benchmarks and BT, eps is fit on the same 1,000 training-split examples used for TS. OE and PDB have no training split, so eps is fit leave-one-out on the evaluation set. Like TS, LS does not change accuracy. Results are for Llama-3-8B; PPT rows are the mean ± SE over 3 seeds.

NLL (lower is better)

ModelOEPDBBTMMLUTruthfulQAHellaSwagARC-CWinogrande
Base7.0934.9102.9121.6913.4901.2610.9781.149
Base + LS4.6154.6241.0981.0861.7380.9620.6890.688
PPT (Dist)4.534 ± .0154.544 ± .0171.253 ± .0051.013 ± .0091.778 ± .0190.880 ± .0140.610 ± .0080.796 ± .004
PPT (Dist) + LS4.534 ± .0144.544 ± .0171.087 ± .0010.975 ± .0041.632 ± .0210.880 ± .0140.601 ± .0060.691 ± .000
Uniform (reference)4.6154.6151.0991.3861.5541.3861.3840.693

ECE (lower is better)

ModelBTMMLUTruthfulQAHellaSwagARC-CWinogrande
Base.467.281.462.196.158.289
Base + LS.041.149.328.075.089.010
PPT (Dist).218 ± .002.112 ± .005.275 ± .007.064 ± .008.058 ± .008.094 ± .007
PPT (Dist) + LS.028 ± .002.091 ± .003.248 ± .008.065 ± .008.049 ± .005.069 ± .003

Smoothing helps the base model substantially, but it does not close the gap to PPT. This suggests that PPT's predictive distributions carry information that neither temperature scaling nor label smoothing of the base model recovers.

The fitted ϵ\epsilon values are also informative. The base model needs heavy smoothing: eps = 0.15-0.30 on MMLU, HellaSwag and ARC-C, and eps is about 1 on OE and PDB, i.e. its numeric answer distribution is no better than uniform. PPT needs little or none: eps <= 0.06 on MMLU, HellaSwag and ARC-C, and eps = 0 on PDB.

AuthorsOpenReview ↗

Continued Response

Implementational detail questions

Q: Were any MCMC diagnostics run?

A: We ran MCMC-NUTS for a portion of the questions, where R-hat and Neff (number of effective sample size) are reasonable. For the dataset, we eventually resorted to rejection sampling instead, as it yields similar results and runs 100x faster on this type of problem. For rejection sampling, we run each program until 1000 Neff is reached.

When MCMC-NUTS converge (R-hat < 1.05), they are very similar with rejection sampling results. This set has 1663 scenarios, NUTS and rejection sampling posterior means correlate at Pearson r = 0.9995 and standard deviations at r = 0.997.

Q: How are OpenEstimate and posteriordb posed and scored? For the multiple-choice tasks, are probabilities normalized over the answer options, and on which data was the temperature fitted? After temperature scaling, Winogrande NLL (0.80-1.03) is worse than a coin flip of 0.69.

A: OpenEstimate. Each query is posed to return <mean> <std> as integers. Proportions are asked on a 0-100 percentage scale. For example, 0 = 30,805 USD, 50 = 125,686 USD, 100 = 220,567 USD. We read the model's distribution over the 101 integers. MAE is the distance between the expectation of that distribution and the true value. NLL is -log of the probability the model gives the rounded true value.

PosteriorDB (PDB). 72 scalar queries on six posteriors (eight schools, surgical, seeds, dugongs, kilpisjarvi,mesquite): 45 queries on posterior means and 27 queries on posterior probabilities of an event (e.g., P(theta_A > theta_C)). The prompt states the Bayesian model in prose (likelihood and priors), lists the observed data, and asks one query on a stated 0-100 grid, in the same answer format. Gold answers come from PosteriorDB's own published programs.

Winogrande NLL above ln 2. One reason is we computed probabilities with a softmax over all 26 letters A to Z. We tried renormalizing over the two options and refitting T, all temperature-scaled models fall below ln 2 (Llama-3: Base + TS 0.685, PPT + TS 0.668; Qwen-2: 0.648 / 0.637), with PPT still ahead. The uncalibrated rows remain above ln 2 because of genuine overconfidence on a task that is also near-chance for these base models.

Q: Which LLM generated the scenarios and programs, what fraction of programs failed or were filtered out, and is there a test split separate from the one used for tuning and early stopping?

A: Yes we use train, validation, and test. Validation and test contain two motifs that are unseen during training, which has the rest 14 motifs. We then evenly divide validation and test, each containing the same number of queries. The bar chart in Figure 2 indeed reports validation numbers. We also ran the same models on test queries, yielding a very small difference that doesn’t affect rankings: sports MAE 5.68 -> 5.71, healthcare MAE 6.44 -> 6.52 (PPT-All Domain). The healthcare MAE reported for PPT-Sports is already completely held-out.

Gemini-3.1-pro generated the scenarios and programs. For the program generations, 1.8% scenarios yielded no executable programs, and these are automatically left out. 12.1% of programs didn’t finish its rejection sampling of 1000 effective sample size in-time (2 CPU hours), and these datapoints are also excluded. We will clarify these designs and processes in the appendix.

Area_Chair_Nz71OpenReview ↗

Engaging with the authors

Dear reviewer,

the authors have posted a reply to your review. Please have a look and interact with the authors as needed.

Thank you, Your AC

These address some of my concerns with the paper, and I thank the authors for running the experiments and disclosing the results earnestly. I still have some doubts when it comes to OpenEstimate, posteriordb or Bayesian teaching against uniform guessing and to me this is important given the claims made about probabilistic reasoning. The label smoothing results did not give what I had hoped to move the paper from borderline. That said, the additional experiment suggests that the model learns more than the label distribution.

I will increase my rating from a 2 to a 3. Though not increase my confidence.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response from Authors

We thank all three reviewers for their thoughtful and constructive feedback. We appreciate their recognition of PPT’s value as a practical approach to scalable probabilistic reasoning supervision and its empirical improvements, including stronger agreement with human judgments. The reviews suggest more precise contribution and generalization claims, additional ablations/controls, and clearer methodological reporting. We believe our clarification on methods or experiments already in the paper do directly answer these related concerns. Additionally-run ablations or controls that match reviewer requests strengthen our claim.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.