acceptodds
Under review as a conference paper at ICLR 2027

Adaptive-GEPA: Make Your Harness Fit Heterogeneous Requests

Abstract

Reflective optimizers such as GEPA improve language-model prompts from execution traces and evaluator feedback; full-program extensions can also rewrite tools and control flow. In practice, a user hands the same endpoint heterogeneous requests whose effective solutions require different tools, reasoning modes, and control flow. Optimizing one shared program leaves this division of work implicit in source-code search, while optimizing a separate program per request family fixes it beforehand. We introduce Adaptive-GEPA, which learns both how to divide requests and how to solve them. It evolves a router and a library of specialist programs under one search budget. The router's instructions, each specialist's description, and its program code are plain, human-readable text, edited from feedback. To combine branches, it aligns specialists by the requests they handle and inherits descriptions together with programs. On a fixed mixture of four task families, the reported Qwen3-8B run evolves four experts without supplying family labels to the router or reflection model; its routing matches the task partition on all 651 test requests. Its family-mean test score () rises from to , compared with for GEPA's full-program adapter and for GRPO at a nominal budget of 18,000 scored calls. Figure 1 summarizes the learning curves, final test scores, and routing agreement.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
reflection optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_bPzdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies program optimization for a fixed mixture of heterogeneous requests without task labels at inference. Adaptive-GEPA jointly edits a natural-language router, specialist descriptions, and executable specialist programs, and uses validation-request overlap to align and inherit description-program pairs across search branches. On HotpotQA, AIME, PUPA, and LiveBench-IF, the authors report family-mean test scores of 70.6 versus 62.5 for single-program GEPA-FPA-ALL on Qwen3-8B, and 71.5 versus 69.8 on GPT-4.1. The search records show changes in task partitioning and cross-branch program reuse.

Critical strengths
  1. The explicit router-description-program representation and footprint-based alignment address the slot-mismatch problem when recombining multi-specialist branches. Figure 7 demonstrates cross-slot alignment, and Appendix H records 10 accepted merges that inherited active specialist pairs unique to both parents.
  2. The full system improves the four-family mean over GEPA-FPA-ALL, a single-program baseline that can also learn internal dispatch and multi-stage workflows (Tables 2-3). Adaptive-GEPA scores lower on Qwen LB-IF (85.0 versus 91.4), but reaveraging the same selected candidates without PUPA still leaves advantages of 2.81 points on Qwen and 2.91 on GPT-4.1 (Appendix G.1).
Critical weaknesses
  1. Unclear handling of corrupted archive entries in the GPT-4.1 run. Appendix L (lines 1301-1306) reports zero PUPA vectors for candidates 15-22 during an evaluator outage and a reset merge counter after resumption. The final candidate was generated after recovery, but the paper does not say whether affected entries were repaired before later parent selection or give the actual merge count relative to the original cap. With a reported GPT-4.1 margin of 1.62 points (Table 8), I would like to know how much subsequent search depended on those entries and the extra merges.
  2. Unclear development decisions after viewing test curves. Appendix L (lines 1296-1299) states that a callback evaluated every fifth accepted candidate and that developers could see the curves, although the optimizer selected candidates using validation scores alone. The paper does not state whether viewing these curves led to changes in the algorithm, hyperparameters, or protocol, or whether any final evaluation used a test set that had not informed those decisions.
Questions for authors

Key Questions For Authors

  1. In the GPT-4.1 experiment, were outage-affected entries 15-22 repaired or reevaluated before they could be sampled as parents? What was the actual total merge count compared with the intended cap (Appendix L)?
  2. Were any components, prompts, hyperparameters, or protocol settings changed after inspecting the test callback curves? Did any final evaluation use a test set that had not informed those development decisions (Appendix L)?
  3. Why does GEPA-FPA-ALL use a single global execution profile while Adaptive-GEPA can select profiles per specialist (Section 4.1; Appendix D, lines 767-771)? How much of the gap remains if both methods have the same profile flexibility?
  4. Appendix I.2 projects the aligned children's scores, whereas Table 12 includes a measured historical index child. Could you execute both aligned and index-based children on a few identical parent pairs from the main run, using the same scoring and capacity rules?
  5. PUPA accounts for about 74% of the Qwen difference in the four-family mean (Appendix G.1), but the saved test results lack per-request score vectors. Could you reevaluate the frozen final candidates on PUPA and separately report answer quality, privacy leakage, and empty delegation-request rates?

Additional Comments

  1. Could you briefly explain why GPT-4.1 was chosen as the second actor model?
  2. Please specify the exact merge scheduling rule. Appendix A (lines 649-651) describes the five-example check; how are those examples divided between cases favoring each parent and ties?
  3. GEPA-FPA-ALL's HotpotQA score is 54.0 in Figure 1(b) and 54.1 in Table 2. Please reconcile the two values.
8 comments
AuthorsOpenReview ↗

Thanks for your review (1/n)

We sincerely thank you for your exceptionally thorough and thoughtful reading of both our main text and appendices, and we appreciate your recognition of our router–description–program representation, footprint-based branch alignment, and consistent gains over single-program search. Your detailed questions helped us clarify our experimental logs and strengthen our empirical analyses, and we address each question (Q1–Q5) and additional comment (A1–A3) point by point below.


Q1 & W1: GPT-4.1 Outage-Affected Entries (15–22) and Merge Count

"In the GPT-4.1 experiment, were outage-affected entries 15-22 repaired or reevaluated before they could be sampled as parents? What was the actual total merge count compared with the intended cap (Appendix L)?"

Entries 15–22 were not repaired or re-evaluated, and the run accepted 20 merges compared with the intended cap of 15; importantly, (1) neither entries 15–22 nor the over-cap merges are in the ancestry of candidate 75, so the final GPT-4.1 program reported in our tables inherits no components from them, and (2) all merge evaluations were counted within the same fixed 18,000-call budget, adding no extra evaluation budget.

  1. Outage-affected entries (15–22) are outside candidate 75's lineage: Tracing the archived run log, entries 15–22 remained in the archive with zero PUPA vectors without re-evaluation and were sampled as parents 6 times, but neither they nor any of their four accepted children (30, 39, 55, 77) is an ancestor of candidate 75—so the final program reported in Tables 3 and 8 inherits no code, descriptions, or routing rules from those entries.
  2. Extra merges (#16–#20) added no extra evaluation budget and are outside candidate 75's lineage: Resuming from checkpoints reset the merge counter, resulting in 20 accepted merges vs. the intended cap of 15. Because every merge evaluation is counted toward the same fixed 18,000-call budget, the extra merges did not add any extra evaluation budget; moreover, candidate 75's lineage contains only two merges (the 3rd and the 15th accepted), so none of the five over-cap merges (#16–#20) is in candidate 75's ancestry.

Q2 & W2: Test Monitoring Callback Curves

"Were any components, prompts, hyperparameters, or protocol settings changed after inspecting the test callback curves? Did any final evaluation use a test set that had not informed those development decisions (Appendix L)?"

No components, prompts, hyperparameters, or protocol settings were tuned or changed based on inspecting the test callback curves. Within each run, all optimizer updates and candidate selection used DfeedbackD_{\text{feedback}} and DvalD_{\text{val}} exclusively (Appendix L); the callback logged scores every 5th accepted candidate for passive runtime monitoring, while hyperparameters were fixed as listed in Appendix B and the mechanism updates in Appendix I.1 were driven by validation-set and runtime crash diagnostics rather than test curves. Final evaluations are reported on the fixed held-out test splits of the four benchmarks without a separate post-development benchmark split; because no design or hyperparameter choices were made from the test callback curves, our final evaluation results were not affected by test-curve tuning.

AuthorsOpenReview ↗

Thanks for your review (2/n)

Q3: Why GEPA-FPA-ALL Uses One Global Profile and Same-Profile Evaluation

"Why does GEPA-FPA-ALL use a single global execution profile while Adaptive-GEPA can select profiles per specialist (Section 4.1; Appendix D, lines 767-771)? How much of the gap remains if both methods have the same profile flexibility?"

GEPA-FPA-ALL uses one global profile because an execution profile (fast or deep) is specified once per program and it optimizes a single monolithic program—and locking all four active Adaptive-GEPA specialists to GEPA-FPA-ALL's deep profile preserves our full gap (70.76 vs. 62.47, +8.29 points).

Experiment setup:

  • To isolate the effect of our learned specialist structure from per-specialist fast/deep profile choices, we set all four active specialists in our selected Qwen3-8B Adaptive-GEPA library to deep—matching GEPA-FPA-ALL (as well as the initial program, which uses deep across every task). Specifically, AIME and LiveBench-IF already use deep, and we switch HotpotQA and PUPA from fast to deep at inference time.

Results:

Table R1. Four-family Qwen3-8B test evaluation with all specialists locked to deep (scores ×100\times 100; means from unrounded scores)

Method / Profile ConfigurationSpecialist ProfileHotpotQAAIMEPUPALB-IF4-Family MeanΔ\Delta vs. FPA-ALL
GEPA-FPA-ALL (paper Tables 2 & 8)Global deep54.136.767.891.462.47—
Adaptive-GEPA (paper Tables 2 & 8)deep: AIME, LB-IF; fast: HotpotQA, PUPA58.247.391.885.070.58+8.11
Adaptive-GEPA (All specialists deep)All 4 active specialists deep58.349.389.885.670.76+8.29

Note: all four families were re-evaluated in this pass

  • With all four active specialists running deep, Adaptive-GEPA reaches 70.76 vs. GEPA-FPA-ALL's 62.47 (+8.29 points), closely matching its original 70.58 score (+8.11 points) and confirming that Adaptive-GEPA's improvements come from our search structure and learned specialist library rather than execution-profile differences.
AuthorsOpenReview ↗

Thanks for your review (3/n)

Q4: Executed Adaptive-GEPA's Merge vs. Index-Based Merge on Identical Parent Pairs

"Appendix I.2 projects the aligned children's scores, whereas Table 12 includes a measured historical index child. Could you execute both aligned and index-based children on a few identical parent pairs from the main run, using the same scoring and capacity rules?"

In a two-stage controlled ablation on Qwen3-8B, Adaptive-GEPA's merge improves measured validation scores on identical parent pairs by +2.10 points across conflicting pairs (+1.20 overall) and raises the end-to-end test family mean from 65.10 (no merging) and 66.58 (index-based merging) to 70.58.

Experiment setup:

  • Stage 1 (Same-parent merge execution): We executed both Adaptive-GEPA's merge and index-based merge on 7 representative parent pairs from the main Qwen3-8B run—covering cross-branch slot collisions (P2, P4), inheriting a newly activated specialist slot (P3), avoiding description–program mismatch by keeping a slot's description and program paired (P7), and cases where both parents share the same slot indices without conflict (P1, P5, P6)—evaluating every child on the full validation set under the exact same scoring and capacity rules.
  • Stage 2 (End-to-end search): Across full Qwen3-8B optimization runs, we compare search without merging, with index-based merging, and with Adaptive-GEPA's merging.

Results:

  • In Stage 1 (same-parent merge execution), Adaptive-GEPA's merge improves mean validation score from 64.60 to 65.80 (+1.20 points) across all 7 pairs, and from 64.78 to 66.88 (+2.10 points) across the 4 conflicting parent pairs (Table R2) while matching index-based merge when no slot conflict exists—reliably preventing specialist overwrites and mismatches when combining branches.

Table R2. Measured validation scores (×100\times 100) of Adaptive-GEPA's merge vs. Index-based merge on identical parent pairs

Parent PairStructural Difference Between Assembled ChildrenAdaptive-GEPA MergeIndex MergeΔ\Delta (Ours −- Index)
P1No slot conflict (identical child)64.3564.350.00
P2Slot collision: LB-IF in slot 4 (Parent A) vs. slot 1 (Parent B)68.4263.54+4.88
P3New specialist: Inheriting a newly activated LB-IF slot65.1764.47+0.70
P4Slot collision: LB-IF in slot 4 (Parent A) vs. slot 1 (Parent B)66.5166.49+0.02
P5No slot conflict (identical child)63.1963.190.00
P6No slot conflict (identical child)65.5665.560.00
P7Paired inheritance: Avoiding description–program mismatch in slot 567.4264.63+2.79
Mean (all 7 pairs)3 identical-child pairs + 4 conflicting parent pairs65.8064.60+1.20
Mean (4 conflicting parent pairs)P2, P3, P4, P766.8864.78+2.10
  • In Stage 2 (end-to-end search), enabling index-based merging brings only a modest gain over no merging (65.10 →\to 66.58), whereas Adaptive-GEPA's merging raises the test family mean to 70.58 (and from 58.18 →\to 59.27 →\to 63.50 excluding PUPA; Table R3), confirming that Adaptive-GEPA's merging clearly outperforms both no merging and index-based merging.

Table R3. End-to-end Qwen3-8B comparison across merge configurations (scores ×100\times 100)

Merge ConfigurationValidation MeanTest Family MeanTest Mean (excl. PUPA)
No merging65.7365.1058.18
Index-based merging67.4566.5859.27
Adaptive-GEPA's merging (Ours)74.0370.5863.50
Reviewer_bXAYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Adaptive-GEPA to optimize a router, specialist descriptions, and specialist programs within a single search. It aligns specialists across search branches using the requests they handled and inherits descriptions together with programs. Experiments use a fixed mixture of HotpotQA, AIME, PUPA, and LiveBench-IF. With Qwen3-8B, the method achieves a family-mean score of 70.6, compared with 62.5 for GEPA-FPA-ALL. The paper also reports results with GPT-4.1 and examines how specialists change during search.

Critical strengths
  1. The method optimizes request allocation and specialist programs within a single search, without specifying in advance which task family each specialist should handle. When merging search branches, it matches specialists with similar request coverage based on the validation requests they actually handled, then inherits their descriptions and programs together. This addresses mismatches between specialist indices and responsibilities across branches and avoids errors caused by merging solely by index. It offers a viable design for optimizing systems that handle multiple types of requests.
  2. Although task-family labels are withheld from the router and reflection model, the learned router matches the task-family partition on all 651 test requests. This result is reported with both Qwen3-8B and GPT-4.1, showing that the system can learn a clear division of tasks from the requests.
Critical weaknesses
  1. The comparison with GEPA-FPA-ALL does not separate the effects of specialist structure and execution profiles. GEPA-FPA-ALL can select only one global profile, while Adaptive-GEPA can select fast/deep for each specialist. In the Qwen setting, this also changes the thinking mode, generation temperature, and output limit, making it unclear how much of the gain comes from specialization. The paper also lacks controlled comparisons of no merging, index-based merging, and the proposed aligned merging. The historical replay in Appendix I.2 does not execute the aligned children and cannot replace an end-to-end ablation.
  2. PUPA accounts for about 74% of the overall advantage in the Qwen experiment, but the utility of this improvement remains unclear. The authors note that empty delegation requests can receive high scores because the local response path retains the original query. Although the final program includes redaction and a nonempty fallback, the test exports do not retain per-request results. The empty-request rate and separate changes in answer quality and privacy leakage therefore cannot be checked. The PUPA score of 91.8 alone does not establish how much external delegation has improved.
Questions for authors
  1. Could the authors compare against GEPA-FPA-ALL using the same execution profile and provide ablations of no merging, index-based merging, and the proposed aligned merging, to distinguish the contributions of execution profiles, specialist structure, and merging?
  2. Could the authors reuse validation traces or conduct a small reevaluation, auditing 30–45 randomly selected PUPA validation requests and reporting the proportion of empty or minimally informative delegation requests, answer quality, and privacy leakage?
  3. Why does the advantage shrink from 8.11 points with Qwen3-8B to 1.62 with GPT-4.1? Since the advantages are similar after excluding PUPA (Table 8), is this change mainly due to PUPA, or does a stronger actor have less need for specialization?
6 comments
AuthorsOpenReview ↗

Thanks for your review (1/n)

We sincerely thank you for your insightful and encouraging review, and we especially appreciate your clear recognition of our footprint-aligned branch merging and the clean task division learned across both Qwen3-8B and GPT-4.1 without task-family labels. Your constructive questions guided us to add targeted empirical evaluations and ablations, which we present point by point below.


W1.1 & Q1.1: Clarifying the Contribution of the Search Structure vs. Execution Profiles

"The comparison with GEPA-FPA-ALL does not separate the effects of specialist structure and execution profiles... Could the authors compare against GEPA-FPA-ALL using the same execution profile...?"

Adaptive-GEPA's improvements come from our search structure and learned specialist library rather than execution-profile differences: with all four active specialists locked to GEPA-FPA-ALL's deep profile, Adaptive-GEPA reaches 70.76 (+8.29 over GEPA-FPA-ALL), matching its original 70.58 (+8.11).

Experiment setup:

  • To isolate the effect of our learned specialist structure from per-specialist fast/deep profile choices, we set all four active specialists in our selected Qwen3-8B Adaptive-GEPA library to deep—matching GEPA-FPA-ALL (as well as the initial program, which uses deep across every task). Specifically, AIME and LiveBench-IF already use deep, and we switch HotpotQA and PUPA from fast to deep at inference time.

Results:

Table R1. Four-family Qwen3-8B test evaluation with all specialists locked to deep (scores ×100\times 100; means from unrounded scores)

Method / Profile ConfigurationSpecialist ProfileHotpotQAAIMEPUPALB-IF4-Family MeanΔ\Delta vs. FPA-ALL
GEPA-FPA-ALL (paper Tables 2 & 8)Global deep54.136.767.891.462.47—
Adaptive-GEPA (paper Tables 2 & 8)deep: AIME, LB-IF; fast: HotpotQA, PUPA58.247.391.885.070.58+8.11
Adaptive-GEPA (All specialists deep)All 4 active specialists deep58.349.389.885.670.76+8.29

Note: all four families were re-evaluated in this pass

  • With all four active specialists running deep, Adaptive-GEPA reaches 70.76 vs. GEPA-FPA-ALL's 62.47 (+8.29 points), closely matching its original 70.58 score (+8.11 points) and confirming that the full +8-point advantage comes from our search structure and learned specialist library rather than execution-profile differences.
AuthorsOpenReview ↗

Thanks for your review (2/n)

W1.2 & Q1.2: Controlled Ablations of Specialist Alignment and Merging

"The paper also lacks controlled comparisons of no merging, index-based merging, and the proposed aligned merging. The historical replay in Appendix I.2 does not execute the aligned children and cannot replace an end-to-end ablation... provide ablations of no merging, index-based merging, and the proposed aligned merging, to distinguish the contributions of execution profiles, specialist structure, and merging?"

In a two-stage controlled ablation on Qwen3-8B, Adaptive-GEPA's merge improves measured validation scores on identical parent pairs by +2.10 points across conflicting pairs (+1.20 overall) and raises the end-to-end test family mean from 65.10 (no merging) and 66.58 (index-based merging) to 70.58.

Experiment setup:

  • Stage 1 (Same-parent merge execution): We executed both Adaptive-GEPA's merge and index-based merge on 7 representative parent pairs from the main Qwen3-8B run—covering cross-branch slot collisions (P2, P4), inheriting a newly activated specialist slot (P3), avoiding description–program mismatch by keeping a slot's description and program paired (P7), and cases where both parents share the same slot indices without conflict (P1, P5, P6)—evaluating every child on the full validation set under the exact same scoring and capacity rules.
  • Stage 2 (End-to-end search): Across full Qwen3-8B optimization runs, we compare search without merging, with index-based merging, and with Adaptive-GEPA's merging.

Results:

  • In Stage 1 (same-parent merge execution), Adaptive-GEPA's merge improves mean validation score from 64.60 to 65.80 (+1.20 points) across all 7 pairs, and from 64.78 to 66.88 (+2.10 points) across the 4 conflicting parent pairs (Table R2) while matching index-based merge when no slot conflict exists—reliably preventing specialist overwrites and mismatches when combining branches.

Table R2. Measured validation scores (×100\times 100) of Adaptive-GEPA's merge vs. Index-based merge on identical parent pairs

Parent PairStructural Difference Between Assembled ChildrenAdaptive-GEPA MergeIndex MergeΔ\Delta (Ours −- Index)
P1No slot conflict (identical child)64.3564.350.00
P2Slot collision: LB-IF in slot 4 (Parent A) vs. slot 1 (Parent B)68.4263.54+4.88
P3New specialist: Inheriting a newly activated LB-IF slot65.1764.47+0.70
P4Slot collision: LB-IF in slot 4 (Parent A) vs. slot 1 (Parent B)66.5166.49+0.02
P5No slot conflict (identical child)63.1963.190.00
P6No slot conflict (identical child)65.5665.560.00
P7Paired inheritance: Avoiding description–program mismatch in slot 567.4264.63+2.79
Mean (all 7 pairs)3 identical-child pairs + 4 conflicting parent pairs65.8064.60+1.20
Mean (4 conflicting parent pairs)P2, P3, P4, P766.8864.78+2.10
  • In Stage 2 (end-to-end search), enabling index-based merging brings only a modest gain over no merging (65.10 →\to 66.58), whereas Adaptive-GEPA's merging raises the test family mean to 70.58 (and from 58.18 →\to 59.27 →\to 63.50 excluding PUPA; Table R3), confirming that Adaptive-GEPA's merging clearly outperforms both no merging and index-based merging.

Table R3. End-to-end Qwen3-8B comparison across merge configurations (scores ×100\times 100)

Merge ConfigurationValidation MeanTest Family MeanTest Mean (excl. PUPA)
No merging65.7365.1058.18
Index-based merging67.4566.5859.27
Adaptive-GEPA's merging (Ours)74.0370.5863.50
AuthorsOpenReview ↗

Thanks for your review (3/n)

W2 & Q2: Per-Request PUPA Audit — Privacy-Preserving Delegation vs. Metric Shortcuts

"PUPA accounts for about 74% of the overall advantage in the Qwen experiment, but the utility of this improvement remains unclear... Could the authors reuse validation traces or conduct a small reevaluation, auditing 30–45 randomly selected PUPA validation requests and reporting the proportion of empty or minimally informative delegation requests, answer quality, and privacy leakage?"

Auditing all PUPA test requests on Qwen3-8B confirms that Adaptive-GEPA's gains come from effective privacy-preserving delegation rather than metric shortcuts: it produces 0.0% empty and 100.0% distinct prompts, raising answer quality over local-only answering (94.6 vs. 79.9) while reducing privacy leakage relative to GEPA-FPA-ALL (11.0 vs. 59.6). Beyond PUPA, Adaptive-GEPA also achieves clear improvements on reasoning and retrieval (such as +10.6 on AIME and +4.1 on HotpotQA on Qwen3-8B, and gains across all three non-PUPA tasks on GPT-4.1; paper Tables 2, 3, and 8).

Experiment setup:

  • Rather than auditing only a 30–45 request subsample, we re-evaluated the frozen final Adaptive-GEPA and GEPA-FPA-ALL programs across all PUPA test requests and inspected every delegated prompt sent to the external model, separately measuring the empty-request rate, proportion of distinct requests, answer quality, and privacy leakage.

Results:

Table R4. Per-request PUPA evaluation on Qwen3-8B (scores ×100\times 100)

MethodEmpty RequestsDistinct RequestsAnswer Quality (↑\uparrow)Privacy Leakage (↓\downarrow)PUPA Score (↑\uparrow)
Empty-request baseline (Local model only)100.0%0.0%79.90.090.0
GEPA-FPA-ALL0.0%89.6%95.259.667.8
Adaptive-GEPA (Ours)0.0%100.0%94.611.091.8
  1. High-quality, task-specific delegation requests (0% empty, 100% distinct): Instead of sending empty strings or generic templates to the external model, our learned PUPA specialist rewrites each query to replace sensitive entities (such as personal names, organizations, and locations) with placeholders ([PERSON], [ORGANIZATION], [LOCATION]) while keeping the user's task instructions intact. In contrast, GEPA-FPA-ALL leaves most personal, organizational, and location details exposed.
  2. Higher answer quality than empty requests and much lower leakage than GEPA-FPA-ALL:
    • vs. Empty-request baseline (local model only): Answering locally without external LLM help reaches 0.0 leakage, but answer quality drops to 79.9. Adaptive-GEPA's external delegation raises answer quality from 79.9 to 94.6, demonstrating that its delegated prompts provide genuinely useful context to the external model.
    • vs. GEPA-FPA-ALL: Adaptive-GEPA achieves comparable answer quality to GEPA-FPA-ALL (94.6 vs. 95.2) while reducing privacy leakage from 59.6 to 11.0, raising the overall PUPA score from 67.8 to 91.8.
    • Why decomposition matters under the standard GEPA metric: Following the standard PUPA composite metric used in GEPA (Agrawal et al., 2026; Li et al., 2025)—which weights quality and 1−leakage1 - \text{leakage} equally—the trivial empty-request baseline already scores 90.0 despite low answer quality (79.9), directly illustrating the composite metric's limitation (Section 6) and why decomposing quality and leakage is essential: on the decomposed metrics, Adaptive-GEPA simultaneously improves answer quality by +14.7 over local-only answering (94.6 vs. 79.9) and reduces privacy leakage by −48.6 relative to GEPA-FPA-ALL (11.0 vs. 59.6).
Reviewer_aKXhOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes Adaptive-GEPA, an extension of GEPA that jointly optimizes a request router and a library of specialist programs for heterogeneous tasks. The method allows separate editing of routing instructions, specialist descriptions, and executable programs, and aligns specialists across search branches based on the requests they handle. Experiments on four task families show improvements over single-program GEPA with Qwen3-8B and GPT-4.1, while learning task-specific routing without explicit task labels.

Critical strengths

The paper addresses the practical problem of optimizing heterogeneous requests within a shared system. Jointly optimizing routing and specialist programs, together with request-based alignment across search branches, provides a concrete approach to organizing program search. The experiments show improvements over single-program GEPA on a mixed workload, with specialists learning distinct execution procedures without explicit task labels. Results with a second actor model provide additional, although more limited, support for the approach.

Critical weaknesses

The contribution of the proposed search structure is not sufficiently isolated. The main comparison changes both the optimization structure and the allocation of execution profiles, making it difficult to attribute the reported improvements to the proposed method. Although the paper presents examples of successful specialist alignment and merging, it lacks controlled ablations to establish their individual contributions. The experimental evidence is limited in its robustness and generalizability. The final method is evaluated with a single seed on a fixed mixture of four task families, and much of the improvement over single-program GEPA comes from PUPA, whose evaluation metric has acknowledged limitations.

Questions for authors

Could the authors clarify how much of the improvement comes from the proposed search structure and specialist alignment, rather than differences in execution profiles? Could they also provide evidence that the PUPA gains reflect effective privacy-preserving delegation rather than metric-specific behavior? Please address the remaining concerns raised in the weaknesses above.

8 comments
AuthorsOpenReview ↗

Thanks for your review (1/n)

We sincerely thank you for your thoughtful review and constructive feedback, and we appreciate your recognition of the practical value of jointly optimizing routing and specialist programs with request-based alignment for heterogeneous workloads without task labels. Your suggestions helped us strengthen our empirical evaluation with targeted controlled experiments and ablations, which we present point by point below.


Q1 & W1: Clarifying the Contribution of the Search Structure vs. Execution Profiles

"The main comparison changes both the optimization structure and the allocation of execution profiles, making it difficult to attribute the reported improvements to the proposed method... Could the authors clarify how much of the improvement comes from the proposed search structure and specialist alignment, rather than differences in execution profiles?"

Adaptive-GEPA's improvements come from our search structure and learned specialist library rather than execution-profile differences: with all four active specialists consistent with GEPA-FPA-ALL's deep profile, Adaptive-GEPA reaches 70.76 (+8.29 over GEPA-FPA-ALL), matching its original 70.58 (+8.11).

Experiment setup:

  • To isolate the effect of our learned specialist structure from per-specialist fast/deep profile choices, we set all four active specialists in our selected Qwen3-8B Adaptive-GEPA library to deep—matching GEPA-FPA-ALL (as well as the initial program, which uses deep across every task). Specifically, AIME and LiveBench-IF already use deep, and we switch HotpotQA and PUPA from fast to deep at inference time.

Results:

Table R1. Four-family Qwen3-8B test evaluation with all specialists locked to deep (scores ×100\times 100; means from unrounded scores)

Method / Profile ConfigurationSpecialist ProfileHotpotQAAIMEPUPALB-IF4-Family MeanΔ\Delta vs. FPA-ALL
GEPA-FPA-ALL (paper Tables 2 & 8)Global deep54.136.767.891.462.47—
Adaptive-GEPA (paper Tables 2 & 8)deep: AIME, LB-IF; fast: HotpotQA, PUPA58.247.391.885.070.58+8.11
Adaptive-GEPA (All specialists deep)All 4 active specialists deep58.349.389.885.670.76+8.29

Note: all four families were re-evaluated in this pass

  • With all four active specialists running deep, Adaptive-GEPA reaches 70.76 vs. GEPA-FPA-ALL's 62.47 (+8.29 points), closely matching its original 70.58 score (+8.11 points) and confirming that the full +8-point advantage comes from our search structure and learned specialist library rather than execution-profile differences.
AuthorsOpenReview ↗

Thanks for your review (2/n)

Q1 & W2: Controlled Ablations of Specialist Alignment and Merging

"Although the paper presents examples of successful specialist alignment and merging, it lacks controlled ablations to establish their individual contributions."

In a two-stage controlled ablation on Qwen3-8B, Adaptive-GEPA's merge improves measured validation scores on identical parent pairs by +2.10 points across conflicting pairs (+1.20 overall) and raises the end-to-end test family mean from 65.10 (no merging) and 66.58 (index-based merging) to 70.58.

Experiment setup:

  • Stage 1 (Same-parent merge execution): We executed both Adaptive-GEPA's merge and index-based merge on 7 representative parent pairs from the main Qwen3-8B run—covering cross-branch slot collisions (P2, P4), inheriting a newly activated specialist slot (P3), avoiding description–program mismatch by keeping a slot's description and program paired (P7), and cases where both parents share the same slot indices without conflict (P1, P5, P6)—evaluating every child on the full validation set under the exact same scoring and capacity rules.
  • Stage 2 (End-to-end search): Across full Qwen3-8B optimization runs, we compare search without merging, with index-based merging, and with Adaptive-GEPA's merging.

Results:

  • In Stage 1 (same-parent merge execution), Adaptive-GEPA's merge improves mean validation score from 64.60 to 65.80 (+1.20 points) across all 7 pairs, and from 64.78 to 66.88 (+2.10 points) across the 4 conflicting parent pairs (Table R2) while matching index-based merge when no slot conflict exists—reliably preventing specialist overwrites and mismatches when combining branches.

Table R2. Measured validation scores (×100\times 100) of Adaptive-GEPA's merge vs. Index-based merge on identical parent pairs

Parent PairStructural Difference Between Assembled ChildrenAdaptive-GEPA MergeIndex MergeΔ\Delta (Ours −- Index)
P1No slot conflict (identical child)64.3564.350.00
P2Slot collision: LB-IF in slot 4 (Parent A) vs. slot 1 (Parent B)68.4263.54+4.88
P3New specialist: Inheriting a newly activated LB-IF slot65.1764.47+0.70
P4Slot collision: LB-IF in slot 4 (Parent A) vs. slot 1 (Parent B)66.5166.49+0.02
P5No slot conflict (identical child)63.1963.190.00
P6No slot conflict (identical child)65.5665.560.00
P7Paired inheritance: Avoiding description–program mismatch in slot 567.4264.63+2.79
Mean (all 7 pairs)3 identical-child pairs + 4 conflicting parent pairs65.8064.60+1.20
Mean (4 conflicting parent pairs)P2, P3, P4, P766.8864.78+2.10
  • In Stage 2 (end-to-end search), enabling index-based merging brings only a modest gain over no merging (65.10 →\to 66.58), whereas Adaptive-GEPA's merging raises the test family mean to 70.58 (and from 58.18 →\to 59.27 →\to 63.50 excluding PUPA; Table R3), confirming that Adaptive-GEPA's merging clearly outperforms both no merging and index-based merging.

Table R3. End-to-end Qwen3-8B comparison across merge configurations (scores ×100\times 100)

Merge ConfigurationValidation MeanTest Family MeanTest Mean (excl. PUPA)
No merging65.7365.1058.18
Index-based merging67.4566.5859.27
Adaptive-GEPA's merging (Ours)74.0370.5863.50
AuthorsOpenReview ↗

Thanks for your review (3/n)

Q2 & W3: Per-Request PUPA Audit — Privacy-Preserving Delegation vs. Metric Shortcuts

"much of the improvement over single-program GEPA comes from PUPA, whose evaluation metric has acknowledged limitations... Could they also provide evidence that the PUPA gains reflect effective privacy-preserving delegation rather than metric-specific behavior?"

Auditing all PUPA test requests on Qwen3-8B confirms that Adaptive-GEPA's gains come from effective privacy-preserving delegation rather than metric shortcuts: it produces 0.0% empty and 100.0% distinct prompts, raising answer quality over local-only answering (94.6 vs. 79.9) while reducing privacy leakage relative to GEPA-FPA-ALL (11.0 vs. 59.6). Beyond PUPA, Adaptive-GEPA also achieves clear improvements on reasoning and retrieval (such as +10.6 on AIME and +4.1 on HotpotQA on Qwen3-8B, and gains across all three non-PUPA tasks on GPT-4.1; paper Tables 2, 3, and 8).

Experiment setup:

  • We re-evaluated the frozen final Adaptive-GEPA and GEPA-FPA-ALL programs across all PUPA test requests and inspected every delegated prompt sent to the external model, separately measuring the empty-request rate, proportion of distinct requests, answer quality, and privacy leakage.

Results:

Table R4. Per-request PUPA evaluation on Qwen3-8B (scores ×100\times 100)

MethodEmpty RequestsDistinct RequestsAnswer Quality (↑\uparrow)Privacy Leakage (↓\downarrow)PUPA Score (↑\uparrow)
Empty-request baseline (Local model only)100.0%0.0%79.90.090.0
GEPA-FPA-ALL0.0%89.6%95.259.667.8
Adaptive-GEPA (Ours)0.0%100.0%94.611.091.8
  1. High-quality, task-specific delegation requests (0% empty, 100% distinct): Instead of sending empty strings or generic templates to the external model, our learned PUPA specialist rewrites each query to replace sensitive entities (such as personal names, organizations, and locations) with placeholders ([PERSON], [ORGANIZATION], [LOCATION]) while keeping the user's task instructions intact. In contrast, GEPA-FPA-ALL leaves most personal, organizational, and location details exposed.
  2. Higher answer quality than empty requests and much lower leakage than GEPA-FPA-ALL:
    • vs. Empty-request baseline (local model only): Answering locally without external LLM help reaches 0.0 leakage, but answer quality drops to 79.9. Adaptive-GEPA's external delegation raises answer quality from 79.9 to 94.6, demonstrating that its delegated prompts provide genuinely useful context to the external model.
    • vs. GEPA-FPA-ALL: Adaptive-GEPA achieves comparable answer quality to GEPA-FPA-ALL (94.6 vs. 95.2) while reducing privacy leakage from 59.6 to 11.0, raising the overall PUPA score from 67.8 to 91.8.
    • Why decomposition matters under the standard GEPA metric: Following the standard PUPA composite metric used in GEPA (Agrawal et al., 2026; Li et al., 2025)—which weights quality and 1−leakage1 - \text{leakage} equally—the trivial empty-request baseline already scores 90.0 despite low answer quality (79.9), directly illustrating the composite metric's limitation (Section 6) and why decomposing quality and leakage is essential: on the decomposed metrics, Adaptive-GEPA simultaneously improves answer quality by +14.7 over local-only answering (94.6 vs. 79.9) and reduces privacy leakage by −48.6 relative to GEPA-FPA-ALL (11.0 vs. 59.6).

Comments on the paper

1 comment
Area_Chair_UcPXOpenReview ↗

AC Comment

I noticed that aKXh and bXAY have not yet provided follow-up feedback. As the discussion period is approaching its end, I would appreciate it if both reviewers could take a moment to review the authors' responses and revised manuscript. Please share whether your concerns have been addressed and update your ratings if warranted. Thank you.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.