acceptodds
Under review as a conference paper at ICLR 2027

UPA: Unsupervised Prompt Agent via Tree-Based Search and Selection

Abstract

Prompt agents have recently emerged as a promising paradigm for automated prompt optimization, framing prompt discovery as a sequential decision-making problem over a structured prompt space. While this formulation enables the use of advanced planning algorithms, these methods typically assume access to supervised reward signals, which are often unavailable in practical scenarios. In this work, we propose UPA, an Unsupervised Prompt Agent that realizes structured search and selection without relying on ground-truth (GT) rewards. Specifically, during search, UPA iteratively constructs an evolving tree structure to navigate the prompt space, guided by fine-grained and position-debiased pairwise comparisons from Large Language Models (LLMs). Crucially, as these local comparisons do not inherently yield a consistent global scale, we decouple systematic prompt exploration from final selection, introducing a two-stage framework grounded in the Bradley-Terry-Luce (BTL) model. This framework first performs path-wise Bayesian aggregation of local comparisons to filter candidates under uncertainty, followed by global tournament-style comparisons to infer latent prompt quality and identify the optimal prompt. Experiments across multiple closed-ended and open-ended tasks demonstrate that UPA consistently outperforms existing prompt optimization methods, showing that agent-style optimization can remain highly effective even in unsupervised settings. The code is available at https://anonymous.4open.science/r/ICLR_2027_ID_57215.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Prompt Optimization, Prompt Engineering, Large Language Model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_jsehOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
2(+1)
Read review
Paper summary

UPA optimizes prompts without ground-truth labels: an MCTS-style tree is grown from judge-LLM child-vs-parent comparisons, a five-point Likert comparison is debiased by swapping order, and a two-stage selection picks the final prompt (path-wise Beta posterior + LCB top-K; global round-robin + BTL/MM). It averages 69.3% on five closed-ended benchmarks vs 67.5% for PDO and 66.3% for SPO.

Critical strengths

The method is specified well enough to reproduce: both algorithms, all hyperparameters, the judge and optimizer prompts, and code are provided. The paper reports a same-budget comparison of search organisations (Table 12: breadth-first 66.8, population 64.7, pool-based 65.5, independent-path 66.8 vs UPA 69.3), a control that prompt-optimization papers usually omit. Empirical coverage is also non-trivial: five closed-ended benchmarks, three execution LLMs plus Qwen3.5-2B, three MT-Bench sub-tasks, and a per-phase cost breakdown (Table 7), with Stage II costing only 4.3% of the optimisation budget.

Critical weaknesses
  1. The claim is not supported. The gain over the strongest GT-free baseline is +1.8 average (Table 1, ≈1.5 SE); cross-model margins are +0.3–0.4 (Table 9, with all eleven methods within 89.6–91.4 on GPT-5); open-ended win rates against SPO/PDO are mostly 50.8–58.3%, with ~60 judgments per cell (all 45 values are 1/120 multiples), i.e. within noise. No confidence interval or significance test appears anywhere.

  2. The closest neighbour is missing. DEEVO (Nair et al., 2025), one of the paper's three motivating GT-free methods and a pairwise-debate + Elo selector, appears in no table; overlapping MT-Bench/BBH protocols make a matched-budget run feasible. Its reported MT-Bench win rates over SPO (66.7–85%) are also well above UPA's (50.8–70%), though under a different protocol.

  3. Judge validity is untested. Nothing checks whether judge preferences track true quality although the closed-ended tasks allow it; the only reported correlation (0.618) is Stage-I vs Stage-II, not against truth. The optimization judge is the executor itself, and App. C.9 shows the judge preferring a response whose stated order contradicts the option it selects because it is "clearer".

  4. Reporting. Most baseline numbers are imported from SPO's paper with no budget counts; the "40% of PDO cost" and "SPO at 65.2%" claims are untabulated; the open-ended protocol (number of judgments, tie handling, order randomisation) and the split used for Fig. 3 are unstated; the Venue column is cosmetic; Q(v) is a back-propagated local improvement, not a node-quality estimate.

Questions for authors
  1. Provide confidence intervals and paired tests for Tables 1–5, 9 and 12 — are +1.8 and +0.3–0.4 distinguishable from zero?
  2. Report judge-vs-GT agreement (per-node Spearman, P(selected = GT-best)).
  3. Run UPA under DEEVO's protocol and report the comparison, or explain why it is impossible.
9 comments
AuthorsOpenReview ↗

[1/5] Response to Reviewer jseh

We thank the reviewer for the detailed feedback. We appreciate your positive comments on UPA's reproducibility, the comparison of search strategies under the same budget, and the experiments across tasks and models. Following your suggestions, we have reproduced the baselines, compared UPA with DEEVO, and evaluated the judge's accuracy. We have added these results, along with statistical analyses and evaluation details, to the revised manuscript.


W1, W2, Q1 and Q3: Statistical results and DEEVO comparison

1. Repeated experiments and statistical results

  • Baseline reproduction. Table 1 previously used baseline results from SPO to maintain consistency with that paper. We have also reproduced these methods with three independent runs each. Table 1 in the revised manuscript reports our reproduced results. Given the time and computational cost of repeated experiments, we selected PDO, the strongest baseline in Table 1, for a comparison with UPA over 30 independent runs per method.
  • Statistical reporting. Appendix C.5 (Table 8) of the original manuscript already reported mean ± standard deviation (std) across three independent runs for SPO and UPA using GPT-5. The revised manuscript provides aggregate statistics in Appendix C.5 (Tables 8–12), the full cross-model comparison in Appendix C.6 (Table 13), and search-strategy results in Appendix D.3 (Table 16). The detailed tables in Parts 3–5 report mean ± std for each dataset and Avg., with a 95% Student's t confidence interval (CI) and a p-value for Avg. We use two-sided paired t-tests with matching seeds for accuracy, and two-sided one-sample t-tests against 50% for MT-Bench win rates. The CIs and tests use df = 2 for three runs and df = 29 for 30 runs. A p-value below 0.05 indicates a statistically significant average improvement in these comparisons.
  • Results. UPA achieves the highest average accuracy in the closed-ended comparisons and ablation studies, and its average MT-Bench win rate exceeds 50% against every baseline on all three models. Most average comparisons are statistically significant. For the gains highlighted in your review (1.8 and 0.3–0.4 percentage points), all corresponding comparisons have p < 0.05. In the 30-run comparison, UPA achieves 69.5% average accuracy versus 67.2% for PDO, a 2.3 percentage point gain (p < 0.001). Its mean accuracy is higher on all five datasets (new Table 9 in Appendix C.5 of the revised manuscript).

2. MT-Bench evaluation and additional runs

  • Scoring. For each model, category and baseline, we evaluated ten questions per run. Each response pair was judged three times in each presentation order, yielding 60 judgments per run. Wins, ties and losses received scores of 1, 0.5 and 0, respectively. The reported win rate is the average of these scores, expressed as a percentage. Its smallest increment within each run is exactly 1/120 on the 0–1 scale. We have added these details to Appendix C.3.
  • Independent runs. Table 3 previously reported results from one independent run. We have added two further independent runs. Revised Table 3 reports mean ± std across all three, and Appendix C.5 (Table 10) reports the aggregate win-rate statistics.
  • Judge prompt. Appendix C.3 provides the complete judge prompt. GPT-5 is the evaluation judge.

3. Direct comparison with DEEVO

  • Reproduction. We previously discussed DEEVO without an experimental comparison because its source code is not publicly available. We have now reproduced DEEVO from the paper and evaluated it alongside the other methods under our protocol. The results are included in revised Tables 1, 2, 3 and 13, with an additional open-weight comparison in Table 14.
  • Different protocols. DEEVO uses five of the ten MT-Bench questions per category for optimization and the other five for testing, but does not report the exact split or its seed. Without the code and this split, we cannot reproduce its exact original experiments. Following SPO, we use ten auxiliary optimization questions per category and all ten benchmark questions for testing, as described in Appendix C.1. The execution models also differ. DEEVO's reported 66.7% to 85% win rates over SPO and UPA's previously reported 50.8% to 70% therefore are not directly comparable.
  • Results. UPA's average accuracy is 2.5 percentage points higher than our DEEVO reproduction in Table 1 (p = 0.001), and is higher on all three executors in revised Tables 2 and 13. Its average MT-Bench win rates against DEEVO are 62.7%, 61.8% and 70.4%, each with p below 0.05.

The detailed results are provided in Parts 3–5.

AuthorsOpenReview ↗

[2/5] Response to Reviewer jseh

W3 and Q2: Judge accuracy and model roles

1. Agreement with ground truth

We measure judge agreement with ground truth (GT) on pairs where exactly one answer is correct. If both answers are correct or both are wrong, their reasoning and overall quality can still differ. Their final-answer labels alone do not determine which response is better overall or whether they should tie. The judge favors the correct response in 3,862 of 4,468 eligible judgments (86.44%). Across child nodes, we correlate the local judge reward R(v) with the change in GT accuracy from parent to child. Both use the same five comparison questions for that node. R(v) averages the judge's scores after combining both presentation orders, as described in Section 2.2 and Eq. (7). The Spearman correlation is 0.813. This checks whether higher judge scores accompany larger accuracy gains. The previously reported 0.618 compares Stage I and Stage II rankings. We have added both judge-versus-GT analyses to Appendix C.5 of the revised manuscript.

2. Separate execution and judging roles

The executor generates answers using a candidate prompt. The optimization judge compares two answers using the dedicated prompt in Appendix B.3. Our default configuration uses GPT-4o-mini for both roles, with temperatures of 0.0 and 0.3, respectively. The roles use different prompts and can be assigned different models. Appendix C.8 (Table 11) of the original manuscript already reported the following results with GPT-4o-mini fixed as the executor and GPT-4o fixed as the optimizer:

Optimization judgeExecutorAvg. accuracy (%)
GPT-4o-miniGPT-4o-mini69.3
GPT-4oGPT-4o-mini69.4
Claude-4.5-SonnetGPT-4o-mini69.3

3. The example in Appendix C.9 of the original manuscript

In Appendix C.9 of the original manuscript, the GT answer is option C. Response A selects C, while Response B states that none of the options matches its reasoning and then selects option A, which is wrong. Although A's intermediate ordering is inconsistent with its selected option, the judge's A_BETTER decision favors the response with the correct final answer.


W4: Optimization costs and reporting details

1. Optimization costs and performance

The baseline costs reported in Appendix C.4 (Figure 4) of the original manuscript, except PDO, were taken from SPO. We have now computed the costs from the measured input and output token usage of our own runs. The table below gives these costs and the accuracies from the three-run Table 1 comparison, together with the additional Scaled SPO result.

MetricAPEOPROPromptAgentPromptBreederTextGradSPOScaled SPODEEVOPDOUPA
Optimization cost (USD)9.324.473.015.2515.100.221.424.583.551.41
Avg. accuracy (%)65.066.266.464.364.465.565.266.867.569.3

UPA's optimization cost is USD 1.41, compared with USD 3.55 for PDO, or approximately 40% of PDO's cost. At a comparable budget of USD 1.42, Scaled SPO reaches 65.2% average accuracy, compared with UPA's 69.3%. These costs apply to prompt optimization. The optimized prompts are used directly at test time.

2. Evaluation details and search statistics

  • Open-ended evaluation. Appendix C.3 now specifies the judgment counts, tie handling and evaluation in both presentation orders.
  • Figure 3 split. Figure 3 reports accuracy on the same final test sets used in Tables 1, 2, 4 and 5. The optimization and test partitions are described in Appendix C.1.
  • Q(v) and Stage I. The review's description of Q(v) matches its definition in Section 2.3 and Eqs. (4) and (8). It is the mean of local comparison rewards backpropagated to the node and is used to guide search. Stage I Bayesian filtering addresses the subsequent prompt-selection problem: under the BTL model, it combines edge comparisons along each path to estimate prompt quality relative to the root, accounts for uncertainty, and filters candidates for Stage II. The selection ablation shows that Max Q(v) gives 65.1% average accuracy, Stage I alone gives 68.3%, and the complete procedure gives 69.3%.
  • Venue column. We have removed the Venue column from Table 1.
AuthorsOpenReview ↗

[3/5] Response to Reviewer jseh

Detailed results

Table numbers below follow the original manuscript, except for the explicitly marked new table (revised Table 9). Accuracy and win rates are percentages. Results are mean ± sample std across three independent runs, with 30 runs for the new PDO comparison. Avg. is calculated within each run. CI denotes a 95% confidence interval of the mean, and Ref. identifies UPA as the reference for paired accuracy comparisons. Bold accuracy values indicate the highest means, and bold p-values are below 0.05.

Table 1. Reproduced closed-ended benchmark results

MethodGPQAMATHLIARWSCBBHAvg.95% CI of Avg.p for Avg.
IO40.9 ± 0.540.7 ± 0.665.4 ± 1.576.2 ± 1.089.7 ± 0.862.6 ± 0.4[61.5, 63.7]0.005
CoT41.7 ± 0.846.9 ± 0.464.0 ± 1.878.9 ± 2.388.5 ± 1.064.0 ± 0.5[62.9, 65.1]0.006
RaR38.7 ± 0.341.5 ± 0.551.1 ± 0.681.5 ± 2.097.2 ± 1.362.0 ± 0.5[60.8, 63.3]< 0.001
APE42.6 ± 0.347.2 ± 1.764.2 ± 0.878.5 ± 1.792.3 ± 0.865.0 ± 0.7[63.1, 66.8]0.023
OPRO41.6 ± 0.846.0 ± 1.465.6 ± 2.581.8 ± 1.796.2 ± 1.066.2 ± 0.5[64.9, 67.5]0.016
PromptAgent44.2 ± 0.844.9 ± 1.767.2 ± 0.681.1 ± 1.794.7 ± 0.866.4 ± 0.6[65.0, 67.9]0.010
PromptBreeder38.6 ± 1.547.4 ± 1.161.5 ± 1.277.1 ± 1.797.0 ± 2.664.3 ± 0.7[62.4, 66.2]0.009
TextGrad39.9 ± 0.544.8 ± 2.666.1 ± 1.077.8 ± 2.093.3 ± 2.564.4 ± 0.4[63.5, 65.3]< 0.001
SPO43.6 ± 1.542.2 ± 1.865.9 ± 1.480.0 ± 1.895.7 ± 1.865.5 ± 0.9[63.3, 67.7]0.007
DEEVO43.7 ± 0.845.6 ± 1.066.7 ± 1.280.7 ± 0.797.3 ± 1.866.8 ± 0.6[65.4, 68.2]0.001
PDO43.6 ± 1.347.7 ± 1.867.9 ± 0.482.0 ± 0.796.2 ± 0.867.5 ± 0.2[66.8, 68.1]0.014
UPA45.5 ± 1.152.1 ± 1.268.2 ± 0.382.7 ± 0.798.0 ± 1.069.3 ± 0.4[68.2, 70.4]Ref.

New table. PDO vs UPA over 30 independent runs (revised Table 9)

MethodGPQAMATHLIARWSCBBHAvg.95% CI of Avg.p for Avg.
PDO43.4 ± 1.347.6 ± 1.867.4 ± 0.781.4 ± 1.196.2 ± 1.067.2 ± 0.6[67.0, 67.4]< 0.001
UPA45.5 ± 1.352.4 ± 0.968.2 ± 0.882.9 ± 1.198.3 ± 1.169.5 ± 0.5[69.3, 69.6]Ref.

Table 2. Transfer to other execution models

ExecutorMethodGPQAMATHLIARWSCBBHAvg.95% CI of Avg.p for Avg.
GPT-5SPO82.5 ± 2.394.9 ± 1.478.0 ± 0.998.2 ± 0.898.5 ± 0.590.4 ± 0.7[88.6, 92.2]0.021
GPT-5DEEVO83.5 ± 1.993.4 ± 0.478.2 ± 1.097.1 ± 1.099.3 ± 0.890.3 ± 0.4[89.2, 91.4]0.038
GPT-5PDO83.3 ± 2.295.2 ± 2.278.3 ± 0.697.6 ± 1.099.0 ± 0.590.7 ± 0.6[89.2, 92.1]0.015
GPT-5UPA84.2 ± 2.195.7 ± 1.478.8 ± 0.898.5 ± 0.4100.0 ± 0.091.4 ± 0.6[89.9, 93.0]Ref.
Claude-4.5-SonnetSPO73.7 ± 1.684.7 ± 1.774.1 ± 0.797.3 ± 0.799.8 ± 0.385.9 ± 0.3[85.2, 86.6]0.037
Claude-4.5-SonnetDEEVO75.1 ± 0.884.9 ± 0.673.7 ± 0.696.0 ± 0.798.3 ± 0.885.6 ± 0.4[84.6, 86.6]0.017
Claude-4.5-SonnetPDO74.7 ± 0.685.8 ± 1.074.2 ± 2.697.1 ± 1.099.5 ± 0.586.3 ± 0.6[84.8, 87.7]0.060
Claude-4.5-SonnetUPA75.8 ± 1.186.6 ± 0.274.7 ± 0.898.0 ± 0.7100.0 ± 0.087.0 ± 0.3[86.3, 87.7]Ref.
DeepSeek-V3.2SPO73.6 ± 2.486.3 ± 1.868.2 ± 0.193.1 ± 0.399.3 ± 0.384.1 ± 0.7[82.4, 85.9]0.057
DeepSeek-V3.2DEEVO74.4 ± 1.189.9 ± 1.767.8 ± 1.192.7 ± 0.797.7 ± 0.884.5 ± 0.5[83.3, 85.7]< 0.001
DeepSeek-V3.2PDO75.1 ± 1.787.8 ± 1.568.3 ± 0.993.3 ± 0.799.3 ± 0.884.8 ± 0.8[82.9, 86.6]0.031
DeepSeek-V3.2UPA78.3 ± 1.093.1 ± 0.668.6 ± 0.593.8 ± 0.499.7 ± 0.386.7 ± 0.5[85.4, 88.0]Ref.
Reviewer_npoYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work focuses on unsupervised prompt optimization. The authors propose UPA, an unsupervised prompt agent capable of structured search and selection without relying on rewards derived from ground-truth labels. Experiments across a range of closed-ended and open-ended tasks demonstrate that UPA consistently outperforms existing prompt optimization methods.

Critical strengths

The experimental section is thorough and solid, providing compelling evidence—across multiple benchmark datasets—that the UPA algorithm is more effective than existing methods.

Critical weaknesses

The accuracy of using a large model to evaluate the relative merits of a child prompt versus a parent prompt has not been sufficiently explained.

Questions for authors

Experiments across various closed-ended and open-ended tasks demonstrate that UPA consistently outperforms existing prompt optimization methods—a very thorough piece of work. However, has there been any separate validation of the accuracy with which a large language model (LLM) evaluates the relative quality of a "child prompt" versus a "parent prompt"?

2 comments
AuthorsOpenReview ↗

Response to Reviewer npoY

Thank you for your review and helpful suggestion. We appreciate your positive assessment of our experiments and UPA's performance across multiple benchmarks. Following your suggestion, we have now validated the judge's parent-child comparisons in two ways. We have added both analyses to Appendix C.5 of the revised manuscript.


Validation of parent-child comparisons

1. Agreement with correct answers. We first compare the judge's decisions with ground truth (GT) on response pairs where exactly one answer is correct. The judge favors the correct response in 3,862 of 4,468 judgments, an agreement rate of 86.44%. These pairs provide a clear preference based on final-answer correctness. When both answers are correct or both are wrong, their reasoning can still differ in quality, so final-answer labels alone cannot determine which response should be preferred or whether they should tie.

2. Judge scores and accuracy gains. We then test whether the judge gives higher scores to child prompts that improve on their parents. Each parent and child answer the same five comparison questions. For each question, we combine the judge's scores from both presentation orders, then average across the five questions to obtain the local reward R(v), as described in Section 2.2 and Eq. (7). On those same questions, we calculate the child's GT accuracy minus the parent's GT accuracy. The Spearman correlation between these accuracy differences and the local judge rewards across child nodes is 0.813. Higher judge scores thus tend to accompany larger accuracy gains over the parent.

These results show that the LLM judge tends to favor correct answers and that its scores track accuracy gains from parent to child. The judge can therefore provide useful feedback for prompt optimization without GT labels.

Dear Reviewer npoY,

We are grateful for your careful review and encouraging comments on our experimental results. Your suggestion to validate the judge's comparisons has helped us strengthen the evaluation.

Our rebuttal provides analyses of judge agreement with GT and the relationship between judge scores and accuracy gains in parent-child comparisons. We have added both analyses to Appendix C.5 of the revised manuscript.

With one day remaining before the discussion deadline, could you let us know whether our rebuttal and revisions have addressed your concern, and whether there are any remaining points we could clarify?

Best regards,

The Authors

Reviewer_vp75OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper focuses on unsupervised automated prompt optimization, analyzes the characteristics of various current prompt optimization methods, and proposes UPA, an unsupervised prompt agent via tree-based search and selection. By two phases, UPA realized realizes structured search and selection without relying on ground-truth (GT) rewards. In the two phases of search and selection, UPA introduces fine-grained and position-debiased pairwise comparison, tree-based search with relative with feedback, and latent quality modeling via BTL, these mechanisms ensure the effectiveness of contrastive learning and search, thereby improving the quality of unsupervised prompts. This paper demonstrated the superiority of UPA over related methods by comparing experiments, and analyzed the roles of each element in UPA by ablation experiments.

Critical strengths
  1. The issue of "unsupervised prompting" is novel. Although there are existing related works, the UPA with two-stage framework proposed in this paper has achieved better results. Especially, this paper provides very accurate and detailed descriptions of some technical aspects.
  2. This paper provides sufficient experiments to confirm the advantages of UPA over related works.
Critical weaknesses
  1. Generally, Related Work should be placed in the main text of the paper. Even if it is placed in the appendix, it should be mentioned in the main text at an appropriate place. However, in this paper, Appendix A (Related Work) is not mentioned in the main text.
  2. Figure 2, although labeled as "Overview of UPA", is mainly intended to explain the principle. If the process of unsupervised prompt optimization of UPA (including input, search and selection two-stage, output, etc.) can be presented with a figure, it will be clearer and more intuitive. The algorithm description can be placed in the appendix.
Questions for authors

Strictly speaking, the current agent should consist of modules such as planning, memory, and action, and is not exactly equivalent to LLM. However, the UPA in this paper seems to be merely implemented using LLM. So, how to demonstrate the characteristics of an Agent?

3 comments
AuthorsOpenReview ↗

Response to Reviewer vp75

Thank you for your positive review and helpful suggestions. We appreciate your recognition of the value of studying unsupervised prompt optimization, the detailed technical descriptions, and the experimental evidence that UPA improves over existing methods. Your comments also point to ways to make the workflow and the meaning of prompt agent clearer.


W1. Reference to related work

We have added the following sentence to the Introduction (Section 1) of the revised manuscript to direct readers to the related work discussion:

We provide a detailed discussion of related work and differences between methods in Appendix A.


W2. Complete workflow illustration

Following your suggestion, we have added a complete workflow illustration in Appendix B.4 (Figure 4) of the revised manuscript to complement Figure 2. Using BBH-Navigate as an example, it starts with the initial CoT prompt and unlabeled task queries, follows the tree search and its feedback loop, and shows how Stage I path-wise Bayesian filtering and Stage II global BTL maximization produce the final optimized prompt.


Q1. The meaning of prompt agent in UPA

We use the term prompt agent in the sense established by work such as PromptAgent (Wang et al., 2024), which treats prompt optimization as a sequential decision problem. PromptAgent is GT-based and relies on rewards derived from ground-truth (GT) labels to guide its search. UPA uses LLM-based pairwise feedback to perform structured search and prompt selection without GT rewards. These methodological differences are discussed in our Introduction and Appendix A.2. In UPA, the agent is the complete system that maintains the search state, chooses refinements, evaluates their outcomes and uses the feedback to make subsequent decisions. The LLMs perform specific roles within this system.

  • Planning. The modified MCTS procedure decides which prompt to refine next. It uses accumulated rewards, visit counts and a diversity penalty to allocate the search budget across branches, as described in Section 2.3.
  • Memory. The search tree retains candidate prompts, their parent-child relationships, visit counts and comparison statistics across iterations. These records guide later search decisions and are reused by Stage I to filter candidates.
  • Action and feedback. The optimization LLM generates a refined prompt from the selected prompt and its execution outputs. The executor then produces responses for the parent and child, and the judge compares them on the same questions. The resulting feedback updates the search statistics and affects the next refinement decision.

These functions form a repeated decision process for improving prompts. UPA carries out this process using pairwise feedback without GT rewards, then selects the final prompt through Bayesian filtering and a global BTL comparison. We have added this explanation of UPA's agent roles to Appendix D.2 of the revised manuscript.


Reference

Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo, Jiayou Zhang, Nebojsa Jojic, Eric Xing, and Zhiting Hu. PromptAgent: Strategic planning with language models enables expert-level prompt optimization. In Proceedings of the Twelfth International Conference on Learning Representations (ICLR), 2024.

Thanks for your rebuttal. I have carefully read it.

Dear Reviewer vp75,

Thank you for the time and care you have devoted to reviewing our work. We hope our responses can address your concerns.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.