acceptodds
Under review as a conference paper at ICLR 2027

Uncertainty Quantification for LLM-based Code Generation

Abstract

Prediction sets provide a theoretically grounded framework for quantifying uncertainty in machine learning models. Adapting them to structured generation tasks, in particular, large language model (LLM) based code generation, remains a challenging problem. An existing attempt proposes PAC prediction sets but is limited by its strong monotonicity assumption on risk and single-label classification framework, which severely limits the space of candidate programs and cannot accommodate the multiple valid outputs inherent to code generation. To address these limitations, we propose an approach RisCoSet that leverages multiple hypothesis testing to construct risk-controlling predictions for LLM-based code generation. Given a trained code generation model, we produce a prediction set represented by a partial program, which is guaranteed to contain a correct solution with high confidence. Extensive experiments on three LLMs demonstrate the effectiveness of the proposed method. For instance, compared with the state-of-the-art, our method can significantly reduce the code removal by up to 24.5%, at the same level of risk.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
conformal prediction, uncertainty estimation, large language models, code generation, prediction sets

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vFBkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RisCoSet, a method for uncertainty quantification in LLM-based code generation. It constructs prediction sets as partial programs by selectively removing uncertain AST nodes. Unlike previous PAC-based approaches, RisCoSet uses the Learn Then Test (LTT) framework to relax monotonicity constraints and account for multiple valid solutions. The authors also introduce a selective execution strategy to reduce calibration costs. Experiments across three LLMs and five benchmarks show improved code retention while maintaining the target empirical coverage.

Critical strengths
  1. Well-motivated approach: The paper addresses an important problem in code generation. Combining AST-based prediction sets with LTT is an interesting way to handle multiple valid programs without imposing strict monotonicity constraints.

  2. Promising experimental results: The method consistently removes fewer AST nodes than the baselines while maintaining comparable coverage. The experiments across different models and benchmarks, along with the sampling analysis, support the usefulness of the approach.

Critical weaknesses
  1. Concerns about the theoretical guarantees: The selective execution analysis appears to have some gaps. The confidence bound in Eq. (10) is derived for a fixed threshold, but Algorithm 3 selects the threshold adaptively. Additionally, the proof of Proposition 3.4 relies on an assumption about false-positive labels that is not clearly justified. It is also unclear how the global labeling error bound translates into the per-task guarantee used in the proof.

  2. Limited practical evaluation: The experiments show improved AST-node retention, but this does not necessarily translate into more useful code. A detailed runtime comparison would also be helpful, particularly given the reliance on Z3 optimization. Additionally, the selective execution results in Table 1 use relaxed coverage targets, making the trade-off between code retention and statistical guarantees important to clarify.

Questions for authors
  1. How is the confidence bound in Eq. (10) justified for the adaptively selected threshold in Algorithm 3?

  2. Can the authors clarify the false-positive assumption and the transition from the global labeling error rate to the per-task bound in Proposition 3.4?

  3. What justifies the combined confidence level of (1−γ)(1−δ) in Proposition 3.4?

  4. Could the authors provide an end-to-end runtime comparison, including the cost of calibration and Z3 optimization?

Reviewer_SZrEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RisCoSet, a risk-controlling framework for code generation. The method addresses two limitations of prior PAC-based approaches: it leverages Learn-Then-Test (LTT) to eliminate the nestedness requirement among partial programs, and adopts a multi-label formulation to account for multiple valid solutions in code generation. The paper further introduces selective execution to reduce the cost of calibration. Experiments across multiple benchmarks and base models demonstrate that RisCoSet retains more AST nodes than existing baselines at comparable target risk levels.

Critical strengths
  • Clear motivation. The paper clearly identifies the limitations of existing PAC-based approaches and provides a coherent motivation for the design of RisCoSet.
  • Good empirical performance. RisCoSet consistently achieves favorable results across the evaluated settings, retaining more AST nodes while maintaining comparable target risk levels to the baselines.
Critical weaknesses
  • Length-independent uncertainty budget. The uncertainty budget λ\lambda is shared across programs of different lengths and is not normalized by the number of AST nodes. Since the constraint is based on the cumulative negative log probability of retained nodes, longer programs may be systematically more likely to violate the same budget and thus require more node removal. This may introduce a length-dependent bias in the utility of the resulting partial programs. The paper does not investigate whether node retention varies systematically with program length or consider length-normalized alternatives.

  • Incomplete experimental coverage. Although the paper evaluates three LLMs and three benchmarks, the main results only report three model-dataset combinations: DeepSeek-Coder on HumanEval, Qwen2.5-Coder on MBPP, and Llama3.1 on APPS. Thus, the claim that the method performs better with Qwen2.5-Coder than with DeepSeek-Coder is not sufficiently supported, since the two models are evaluated on different datasets. More complete cross-model and cross-dataset evaluations would be needed to support such conclusions.

  • Limited practical applicability. There are two concerns regarding the practical applicability of the proposed framework.

    First, distribution shift. The theoretical guarantee assumes that the calibration and test data are drawn from the same distribution, whereas this assumption may not hold in realistic deployment scenarios. The reported results under distribution shift further suggest that the method can become substantially less useful: at a 10% target risk, the reported node removal reaches 75.8%, leaving only about 24.2% of the original AST nodes and substantially reducing the utility of the resulting partial programs.

    Second, computational overhead. Constructing the partial programs relies on Z3-based integer optimization. Although the implementation jointly optimizes multiple candidate λ\lambda values, it uses a 24-second solver timeout per optimization instance. The paper does not report solver runtime statistics or timeout rates, making it difficult to assess the scalability of the method.

Questions for authors
  1. Under what assumptions does RisCoSet+ preserve the risk-control guarantee under distribution shift? In Table 6, at α=0.1\alpha=0.1, reweighting improves coverage from 84.5% to 90.1% but increases node removal from 57.5% to 75.8%. Could the authors explain this tradeoff and evaluate whether the retained partial programs remain practically useful?

  2. Does a shared uncertainty budget λ\lambda disproportionately penalize longer programs? Could the authors report node-retention rates across program lengths and evaluate normalizing the cumulative negative log-probability of retained nodes by the original AST size? Would recalibrating this normalized formulation preserve the risk-control guarantee?

  3. Could the authors compare the evaluated models on shared benchmarks to disentangle model effects from dataset effects? In particular, the claim that Qwen2.5-Coder performs better than DeepSeek-Coder requires comparisons on the same datasets. Please also clarify the complete set of model–dataset combinations evaluated.

  4. Could the authors report calibration and deployment runtimes separately, including LLM generation, test execution, and Z3-based partial-program construction? Please provide the fraction of calibration time spent on Z3, mean/median/P95 solver runtimes, timeout rates, and how timed-out instances are handled.

  5. How much end-to-end calibration time does selective execution save? Given that calibration is performed offline and λ\lambda can be reused, how frequently would recalibration be needed in practice, and when would these savings be substantial?

Reviewer_yLG8OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To address the lack of reliable uncertainty quantification signals in large language model code generation, this paper proposes a method named RISCOSET. Existing PAC prediction set methods for code generation are constrained by a strong monotonicity assumption on risk and a single-label classification framework, failing to accommodate the multiple valid correct outputs (multi-label property) naturally present in code generation and the complex candidate program space. Comprehensive evaluations were conducted across three open-source LLMs (DeepSeek-Coder-33B, Qwen2.5-Coder-32B, Llama3.1-70B) and multiple code/SQL generation benchmarks (HumanEval, MBPP, APPS, LiveCodeBench, Spider). Experiments demonstrate that under the premise of satisfying target risk control, this method can significantly reduce AST node pruning.

Critical strengths
  1. Previous methods (such as PAC prediction sets) heavily rely on the single-label assumption and strict monotonicity constraints, which conflict with the nature of "multi-solution" (multiple syntactically different but semantically correct programs) in code generation tasks. This paper cleverly introduces the LTT framework into code generation, transforming the risk control objective into structured semantic correctness (ensuring the prediction set contains at least one correct program) and breaking free from stringent monotonicity assumptions. It exhibits remarkable superiority in experiments—substantially reducing unnecessary code pruning while maintaining compliant coverage.
  2. Sampling a large number of candidate programs for each task in the calibration set and running test-case verification incurs a heavy computational burden. The selective execution strategy designed in the paper can safely skip a large proportion of unnecessary test executions without compromising the overall theoretical guarantees of risk control.
Critical weaknesses
  1. The calibration and enhanced label set construction of this method heavily rely on large-scale test-case execution of the generated code. Although the authors mention in the appendix that this method can be extended to text-to-SQL or other scenarios with deterministic evaluation environments, obtaining reliable S_i is extremely difficult or even infeasible in many real-world software engineering scenarios that lack complete automated test cases or depend on complex external interactions.
  2. In the optimization and solving phase for obtaining partial programs, the authors use the Z3 solver and set a 24-second timeout limit for each instance. For extremely long, complex functions or massive repository-level codebases, the overhead of constraint solving to find the optimal subtree or satisfy structural constraints (SC1, SC2) may expand sharply, leading to solver timeouts or performance bottlenecks.
  3. Lack of related decoding baselines.
Questions for authors
  1. Regarding the impact of test-case completeness on risk guarantees: During the calibration stage, multi-label augmentation and selective execution rely on test cases to determine whether the generated code is "correct". If the unit test cases in the benchmark datasets themselves have vulnerabilities, are incomplete (coverage deficiency), or suffer from "overfitted tests," what kind of systematic bias would this impose on the empirical structured risk and the ultimate risk control guarantees of the LTT framework?
  2. Regarding the scalability of the SMT solver on large-scale code: The experiments mention using the Z3 solver to solve the optimization problem satisfying AST structural constraints (SC1, SC2) with a 24-second timeout. When facing complex repository-level code containing hundreds of nodes, what are the success rate and average latency of Z3? Have the authors considered designing greedy or heuristic alternative solving strategies to replace the costly SMT solver?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.