acceptodds
Under review as a conference paper at ICLR 2027

Planning to Explore: Curiosity-Driven Exploration for LLM Test Generation

Abstract

The use of LLMs for code generation has naturally extended to code testing and evaluation. As codebases grow in size and complexity, so does the need for automated test generation. Current approaches for LLM-based test generation rely on strategies that maximize immediate coverage gain, a greedy approach that plateaus on code where reaching deep branches requires setup steps that individually yield zero new coverage. Drawing on principles of Bayesian exploration, we treat the program's branch structure as an unknown environment and maintain an exploration state built from execution, represented by a coverage map and a per-function posterior over which parts of the program the agent can reach. Our method, CovQValue, uses this posterior to target candidate test plans at unexplored code, asks the LLM which functions each plan will execute and enable, and selects the plan with the highest estimated Q-value. We show that the major gains come from using the exploration state to decide what to explore next. Our method outperforms greedy selection on TestGenEval Lite, achieving 103-150% higher branch coverage and winning on 83-88% of targets. In addition, we build a benchmark for iterative test generation, RepoExploreBench, where CovQValue achieves 129-157% higher branch coverage than greedy selection across three LLMs, winning on 94-97% of targets. These results show the potential of curiosity-driven exploration for LLM agents, enabling more effective discovery with sequential interaction.

Primary area:
generative models
Keywords:
curiosity-driven exploration, test generation, code coverage, bayesian exploration, planning, llms

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1zqKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposesa method, namely CovQValue, for iterative LLM-based test generation inspired by Bayesian exploration. It uses coverage information and per-function reachability estimates to guide test-plan generation and selection. The paper also introduces RepoExploreBench and evaluate CovQValue on two benchmarks using three LLMs. The results show substantial improvements in branch coverage over greedy baselines.

Critical strengths
  • The paper addresses a relevant problem in LLM-based test generation.
  • The proposed method achieves substantial coverage improvements across two benchmarks.
  • The evaluation includes three LLMs and useful ablation studies.
Critical weaknesses
  • The contribution of future-value estimation is not supported by the ablation results. A central motivation of the paper is to account for future exploration opportunities. However, Figure 5(c) reports slightly higher mean coverage when the future-value term is removed ((γ\gamma=0)), compared with the default configuration (88.6 versus 85.5). The component ablation shows a similar result, with coverage changing from 86.5 to 85.5 after adding the Bayesian future-value term. These differences do not establish that the term is harmful, but neither experiment demonstrates a clear benefit. Most of the reported improvement comes from targeting unexplored functions. While multi-step planning may still be useful, the specific contribution of future-value estimation remains unsubstantiated.

  • The probabilistic interpretation of the reachability estimates is unclear. The method maintains a Beta posterior to estimate each function's reachability. Equation (3) assumes that a plan increases the future reachability of functions it enables, even if they are not immediately executed. However, Algorithm 1 increases the posterior's failure count whenever a predicted function is not covered. This also applies to functions predicted to be enabled but not executed. Their estimated reachability therefore decreases, despite the future-value calculation assuming an increase. These mechanisms may represent different aspects of reachability, but the paper does not clearly distinguish them. As a result, the probabilistic meaning of the exploration state remains uncertain.

  • The function-level exploration objective does not fully align with the branch-level coverage goal. Algorithm 1 defines unexplored functions as those for which no line has been executed. Equation (3) then estimates exploration value using this set of functions. Once a function has been executed, it is excluded from the explicit exploration objective, even if many of its branches remain uncovered. This is important because reaching deeper branches is a central motivation of the paper. Although the coverage map may still help the agent discover such branches, the scoring mechanism does not explicitly account for them. The gap between function-level targeting and branch-level coverage is therefore not fully addressed.

  • The empirical evidence for the incremental contribution over existing coverage-guided methods is limited. The main Greedy baseline does not receive explicit coverage feedback, while CovGreedy receives the coverage map but selects candidates randomly. CovQValue additionally uses function targeting, multi-step plans, and candidate scoring. The component ablation helps separate these effects and shows that targeting accounts for most of the improvement. However, coverage-guided targeting is already used in existing approaches such as CoverUp. The comparison with CoverUp uses each tool's native budget, making the results difficult to interpret as a controlled comparison. Moreover, the advantage over CoverUp is relatively small for Gemini 3.8 Flash (103.5 versus 98.0 branches). The evidence for an additional benefit over strong existing approaches is therefore less convincing than the comparisons with the basic greedy baselines suggest.

Questions for authors
  • What evidence demonstrates the benefit of future-value estimation beyond targeted immediate-gain scoring?
  • What probability does the per-function Beta posterior represent?
  • How does CovQValue account for uncovered branches within previously executed functions?
  • How does CovQValue compare with CoverUp under matched execution and inference budgets?
  • How is the fault-detection effectiveness of the generated scripts assessed?
2 comments

In the following response, we tried to answer your questions. The reported numbers are mean covered branches per target on RepoExploreBench (93 targets, 24 executions) unless stated. ± is the 95% confidence margin. And counts are slightly lower than in the paper because we now count only true branch arcs.

W1 / Q1: Benefit of future-value estimation

We compared the method with and without the future-value term on RepoExploreBench, and on synthetic programs built to require lookahead (a chain of 4–8 functions that must be called in order across separate test scripts, with most branches behind the last one). There we know what each plan truly unlocks and can substitute it for the LLM's prediction (60 paired runs). On these programs, every compared method keeps a function as a target until all of its lines have run.

Effect of adding the future-value termDifference
RepoExploreBench, LLM-predicted values−1.3 ± 1.9
Synthetic programs, LLM-predicted values−1.1 ± 6.2
Synthetic programs, true values+13.6 ± 5.9

With the true downstream values, the term adds 13.6 branches. With the LLM's predicted values the effect is neutral, which matches Figure 5(c). The bottleneck is prediction quality. We show this bottleneck and show how we can potentially reach better results with our method with better prediction capabilities.

W2 / Q2: What the Beta posterior represents

pfp_f is the probability that a function the LLM expects a plan to execute really gets executed. The inconsistency you describe comes from functions predicted as "enabled", which Algorithm 1 penalized when they were not executed right away. We removed that penalty, so only execution predictions update pfp_f. Coverage barely moves (−0.4 branches on Gemma, 3 seeds), so the results in the paper do not depend on it. We update Algorithm 1 and state this definition.

W3 / Q3: Uncovered branches inside executed functions

We added hybrid targeting. In each round, one candidate plan revisits the executed functions with the most uncovered branches, and its prompt lists those branches.

Hybrid targeting vs. submitted method (Gemma)Difference
RepoExploreBench, 24 executions (3 seeds)+1.7 ± 2.4
RepoExploreBench, 48 executions (1 seed)+5.1 ± 2.9
Synthetic programs, vs. remaining-targets variant (60 runs)+9.6 ± 5.9

The gain grows once the unexecuted functions run out, and we will adopt hybrid targeting in the revision. A function-level breakdown of where the branch gains come from is in our response to Reviewer jDfu (W3 / Q3).

W4 / Q4: CoverUp under matched budgets

We reran CoverUp with a cap of 48 LLM calls and 24 executions per target, the budget CovQValue uses.

Model and budgetCoverUp calls / executionsCoverUpCovQValueDifference
Gemma, native budget126 / 8943.581.4+37.9 ± 15.8
Gemma, matched budget31 / 2427.781.4+53.7 ± 16.3
Gemini, matched budget48 / 746.199.7+53.6 ± 16.6

Under the same budget, CovQValue covers more than twice as many branches on both models, while at its native budget CoverUp needs 2.6 times our LLM calls. CoverUp handles uncovered segments one at a time in file order, and on 55 of 93 targets (Gemma) it ends the matched budget with no passing test. CovQValue instead spends each execution on the plan with the highest expected gain.

Q5: Fault detection

We measured fault detection with mutation analysis. For each of 30 RepoExploreBench modules, sampled across its 9 repositories, we created 50 faulty versions with one small change each. We then ran each tool's tests (generated with Gemma) and counted how many faulty versions they detect. Our scripts have no assertions, so a fault counts as detected when a script's output or exception changes.

Tests fromFaults detectedDetected, among faults in code the tests execute
CovQValue47.9%71%
CoverUp34.6%79%
Pynguin8.8%38%
CovGreedy9.4%73%

When a tool produced no tests for a module, it detects no faults there (6 modules for CoverUp, 14 for Pynguin).

CovQValue detects far more faults than Pynguin and CovGreedy, and is on par with or better than CoverUp's assertion-based tests. Output comparison and assertions are different checks, so that last comparison is approximate. The higher coverage does translate into more faults detected.

As we are close to the end of the discussion period, we wanted to check if our response addressed your concerns. We went over the future-value evidence, the posterior definition, the matched-budget CoverUp comparison, and the mutation analysis. We would appreciate if you can reconsider your score in light of the new results, and we remain available for any additional question.

Reviewer_jDfuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies iterative test generation with large language models, aiming to improve branch coverage under a limited execution budget. The proposed method, CovQValue, draws on Bayesian exploration. It maintains a coverage history and function-level reachability estimates, generates candidate test plans for functions that have not yet been executed, and selects plans based on both expected immediate coverage gains and the value of future exploration. The authors also introduce RepoExploreBench, which contains 93 modules from 9 Python packages, and evaluate the method with three large language models on this benchmark and TestGenEval Lite. The full method achieves higher average branch coverage. The ablations suggest that most of the gain comes from directing plan generation toward unexecuted functions.

Critical strengths
  1. Explicit exploration state guides test generation. The method turns execution feedback into a coverage history and function-level reachability estimates, then uses this information to generate and select test plans. I find this direction worthwhile. It provides a concrete way to use accumulated execution information to guide further exploration of uncovered code, and the overall results suggest that it can improve branch coverage.

  2. RepoExploreBench provides a useful benchmark for complex real-world code. It includes 93 modules from 9 Python packages, totaling approximately 77,000 lines of code, with a shared containerized execution environment. Its focus on object construction, configuration dependencies, and nested calls makes it a useful resource for evaluating how iterative test generation explores complex programs.

Critical weaknesses
  1. The function-level exploration state may limit further exploration of branches within a function. Branch coverage is the main evaluation metric, but reachability estimates and plan scores are defined at the function level. According to Algorithm 1, executing any line in a function removes it from the set of unexecuted functions, even if many of its branches remain uncovered. The scoring mechanism therefore cannot distinguish a function whose simple entry path has been covered from one that has been thoroughly explored. It also does not explicitly estimate the benefit of revisiting the former.

  2. The external baselines do not adequately cover relevant approaches. The paper compares against only Pynguin and CoverUp, leaving recent program-analysis-assisted methods and iterative test-generation agents underrepresented. For example, TestWeaver uses program slicing and execution feedback to address coverage stagnation, Test4Py uses call graphs and type inference to improve test input construction, and TestForge iteratively refines tests using execution results and coverage reports. These methods are directly relevant to the paper's task and motivation.

  3. The coverage analysis does not explain how the function-level exploration mechanism contributes to the gains. The method maintains reachability estimates and scores candidate plans at the function level, yet the experiments mainly report branch coverage. These results do not sufficiently distinguish where the gains come from. I would like to see function/method coverage, defined as the proportion of target functions reached, together with an analysis of branch coverage within those functions.

Questions for authors
  1. Algorithm 1 removes a function from the unexecuted set as soon as it is reached. How does the method then guide tests toward uncovered branches within that function? Does this design affect exploration of functions with high cyclomatic complexity?

  2. TestWeaver, Test4Py, and TestForge also use program analysis or iterative execution feedback to improve test coverage. Why were these methods excluded from the comparison? Where feasible, could the authors compare against representative methods under comparable budgets to clarify the relative advantages of CovQValue?

  3. Could the authors report function/method coverage and separate branch coverage gains in newly reached functions from gains in previously reached functions? This would help establish whether the method mainly reaches more functions or also explores individual functions more deeply.

2 comments

We ran multiple experiments in this response. Numbers are on RepoExploreBench. ± is the 95% confidence margin. Counts are slightly lower than in the paper because we now count only true branch arcs.

W1 / Q1: Uncovered branches inside functions that were already reached

In the submitted method, plans already return to reached functions, and about a fifth of new branches come from them (table under W3). To guide this explicitly, we added hybrid targeting. In each round, one candidate plan revisits the reached functions with the most uncovered branches, and its prompt lists those branches.

Hybrid targeting vs. submitted method (Gemma)Difference in branches
RepoExploreBench, 24 executions (3 seeds)+1.7 ± 2.4
RepoExploreBench, 48 executions (1 seed)+5.1 ± 2.9
Synthetic programs, vs. remaining-targets variant (60 runs)+9.6 ± 5.9

Hybrid targeting did not hurt in any setting we tested, and it helps once the unreached functions run out, so we will adopt it. Complex functions are harder for every method. From the simplest third of functions to the most complex third, coverage inside a function falls from 84% to 69% for CovQValue and from 80% to 71% for CovGreedy (Gemini).

W2 / Q2: Additional external baselines

We added Test4Py, the one of the three that we could run on our local model within the rebuttal period. We ran it on 30 stratified targets (Gemma), with a one-hour limit per target.

30 targets, GemmaBranches coveredLLM calls per target
Test4Py66.9366
CovQValue88.848

Over all 30 targets, CovQValue covers more branches with about one seventh of the LLM calls. Test4Py covers more on the 18 targets it completes (72.4 vs 48.6), and fails to finish on the other 12. We found no released code for TestWeaver and TestForge requires a complex setup for our experiment.

W3 / Q3: Function coverage, and where the branch gains come from

We replayed every run in the paper and assigned each covered branch to its function. A function counts as reached once any of its lines has run.

ModelFunctions reachedBranch coverage inside reached functionsCovQValue's new branches from previously reached functions
Gemini 3.8 Flash78% → 94%76% → 78%24%
Gemma 4 31B18% → 83%88% → 66%19%
GLM 5.350% → 93%75% → 76%22%

Arrows show CovGreedy → CovQValue. On Gemma, CovGreedy reaches only a few simple functions, which is why its within-function coverage is high.

CovQValue gets tests into 83–94% of the functions in a module, against 18–78% for CovGreedy, and its extra branches are in those newly reached functions. Inside reached functions the two methods cover a similar share of branches, and about a fifth of CovQValue's new branches come from functions it had already reached.

So the gains come mainly from reaching more functions, and each function is still explored as deeply as with CovGreedy.

As we reach the end of the discussion period, we wanted to check if our response addressed your concerns. We covered hybrid targeting for within-function branches, the added Test4Py baseline, and the function-coverage analysis. We would appreciate reconsidering your score, and we’re happy to answer any remaining questions.

Reviewer_ZakbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors examine an important challenge in iterative test generation using large language models with the aim of improving branch coverage for complex code under a constrained execution budget.

Critical strengths
  • The authors introduce CovQValue, a method inspired by traditional Bayesian exploration. It selects execution plans by trading off immediate coverage improvements against the potential value of future exploration.

  • In addition, the authors develop the RepoExploreBench benchmark, which contains 9 Python packages and 93 modules, and then evaluate it using three large language models.

  • Experimental results indicate that compared with conventional greedy strategies, CovQValue substantially improves branch coverage.

Critical weaknesses
  • While the idea is novel, it is unexpected to find that incorporating future value terms does not appear to provide promising benefits. When these terms are included, the average number of covered branches is lower, and it is even below the performance achieved without them. This result suggests that relying solely on immediate return terms may produce the same outcome, raising the possibility that non-myopic Bayesian planning is unnecessary here.

  • It is not convincing how the probabilistic interpretation of the β posterior updates. After a function has been executed, it is removed from the candidate set. Therefore, applying a forward posterior update at that stage has almost no impact on the scores of subsequent candidates.

  • The Q-value scoring mechanism also appears to suffer from selection bias. CovQValue generates three candidate plans in each round, while the main decision-making process depends on ranking their values. Nevertheless, the authors assess prediction accuracy only for the selected plans and do not examine ranking accuracy across all candidate plans.

  • Additionally, it would be better to include multi-seed runs. Since the absence of random seeds suggests that the experimental findings may be affected by chance. The authors tested random seeds only under the particular setting of Gemma 4 31B.

Questions for authors
  • Can the authors clarify which probabilistic events are represented by the β posterior distribution? After a function has been executed and removed from the candidate set, how would a positive posterior update influence later exploration decisions?

  • Can the authors provide evidence demonstrating that the future value term is beneficial? Please elaborate on its practical utility.

  • Can the authors assess whether the ranking of Q-value scores is consistent with the actual rewards obtained by all candidate plans?

2 comments

We ran three new experiments. Numbers are mean covered branches per target on RepoExploreBench (93 targets, 24 executions). ± is the 95% confidence margin on paired differences. Counts are slightly lower than in the paper because we now count only true branch arcs.

Q2 / W3: Baselines with the same target information

We reran Random, Greedy and CovGreedy with the same target snippets and instruction that CovQValue gets ("+ targets"). We also added a control that generates CovQValue's plans but picks one at random.

MethodLLM calls per targetGemma 4 31BGemini 3.8 Flash
CovGreedy (paper)7232.890.8
Greedy + targets9668.790.6
Random + targets7275.792.9
CovGreedy + targets7278.6104.7
CovQValue with random plan selection4877.397.3
CovQValue4881.499.7

The targets come from our method, which tracks the functions no test has reached yet and passes them to the generator. The "+ targets" rows show that this component also lifts the baselines. CovGreedy goes from 32.8 to 78.6 on Gemma and from 90.8 to 104.7 on Gemini.

Q-value selection then adds a further 4.1 branches on Gemma and 2.3 on Gemini over picking one of the same plans at random.

CovQValue also uses the fewest LLM calls. If CovGreedy + targets is stopped at the same 48 calls, it reaches 74.5 on Gemma and 100.0 on Gemini, so CovQValue is ahead on Gemma and level with it on Gemini.

Q1 / W1: Independent contribution of the future-value term

We compared the method with and without the future-value term on RepoExploreBench, and on synthetic programs built to require lookahead: a chain of 4–8 functions that must be called in order across separate test scripts, with most branches behind the last one. There we know what each plan truly unlocks and can substitute it for the LLM's prediction (60 paired runs). On these programs, every compared method keeps a function as a target until all of its lines have run.

Effect of adding the future-value termDifference
RepoExploreBench, LLM-predicted values−1.3 ± 1.9
Synthetic programs, LLM-predicted values−1.1 ± 6.2
Synthetic programs, true values+13.6 ± 5.9

With the true downstream values, the term adds 13.6 branches, so looking ahead does help beyond targeting and immediate-gain scoring. With the LLM's predicted values the effect is neutral, as in the paper's ablation. The bottleneck is prediction quality. On the synthetic programs, the LLM flags 64% of the plans that do unlock something, but also 33% of those that don't. We will add this study to the paper.

W2: Validity of the posterior update rule

Thanks for pointing these out. We have fixed both. When a predicted function gets covered, that success now also raises our estimate for the functions that are still uncovered, and a function predicted as "enabled" is no longer penalized just because it was not executed right away. Coverage barely moves with the new rule (−0.4 branches on Gemma, 3 seeds), so the results in the paper do not depend on it. We will update Algorithm 1 and state clearly what pfp_f means: the probability that a function the LLM expects a plan to execute really gets executed.

As the discussion period reaches the end, we wanted to check whether our response addressed your concerns. It covers new evidence for the future-value term, the corrected posterior, and a plan-selection control. If so, we would appreciate you considering an updated score, and we’re happy to answer any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.