acceptodds
Under review as a conference paper at ICLR 2027

GuidedEvolve: Feedback-Guided Prompt Evolution for Discovering Implicit Decision Criteria

Abstract

Reliable decision-making with Vision-Language Models (VLMs) often requires adherence to implicit, domain-specific decision policies that go beyond generic reasoning instructions. Prompt optimization provides a practical way to elicit such behavior, yet existing methods largely rely on stochastic search or scalar performance feedback, offering limited insight into why a prompt fails or how it should be improved. To address this limitation, we propose GuidedEvolve, a feedback-guided prompt evolution framework for decision-making that transforms case-level feedback analysis into actionable decision policies. GuidedEvolve identifies fine-grained failure patterns, distills them into transferable decision strategies, and uses these strategies to guide subsequent prompt evolution, turning prompt optimization from metric-driven exploration into interpretable and directed refinement. We evaluate GuidedEvolve on multimodal misinformation detection across public benchmark datasets. Experimental results show that GuidedEvolve consistently outperforms existing prompting and prompt-optimization methods, demonstrating the effectiveness of explicit feedback-guided evolution for complex multimodal decision-making.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Misinformation Detection, Automatic Prompt Optimization, Prompt Evolution, Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_xRgjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GuidedEvolve, a prompt-evolution framework for multimodal misinformation detection. At each iteration, an analyst VLM examines a batch of validation cases and generates structured diagnoses covering the error type, overlooked evidence, likely source of confusion, and a proposed revision. The framework accumulates these suggestions and uses them to produce a feedback-guided prompt alongside a standard evolutionary candidate; the candidates are compared using validation performance. The evaluation covers MMFakeBench, AMG, and DeceptionDecoded, with additional backbone comparisons on AMG. GuidedEvolve reports the best F1 among the compared methods on all three datasets, with the largest margin over GEPA on MMFakeBench and smaller margins on AMG and DeceptionDecoded. Ablations suggest that both diagnostic feedback and the guided-candidate branch contribute to the final performance.

Critical strengths

1. A relevant problem and a modular optimization design.

The paper recognizes that heterogeneous failures may require different prompt revisions, while aggregate fitness provides little guidance about what to change. Using case-level diagnoses to guide one branch of an evolutionary search is a reasonable and practically deployable idea. It does not require fine-tuning the task VLM, and in principle it can be integrated into existing prompt optimizers.

2. Broad baseline coverage and component ablations.

The evaluation spans three multimodal misinformation benchmarks and compares against a range of prompt-optimization approaches. The ablation removing diagnostic feedback performs worse than the full method, providing preliminary evidence that feedback contributes beyond unguided candidate generation. The effort to keep total LLM-call budgets close also makes the comparison more informative than an evaluation with unconstrained extra search.

Critical weaknesses

1. The claimed discovery of reusable decision criteria is not directly demonstrated, and the distinction from existing reflective optimizers remains unclear.

The method adds structured error diagnosis to an evolutionary optimizer, but the paper does not isolate which part of this structure is novel relative to natural-language reflection in methods such as GEPA and PromptAgent. The accumulated feedback is described largely as a set of suggestions; it is not clear how duplicate or contradictory suggestions are consolidated into stable rules, or how a rule inferred from one case is distinguished from an overfit correction for that case. The reported F1 improvements demonstrate utility of the full pipeline, but do not by themselves demonstrate that it discovers interpretable, reusable, or transferable decision criteria.

2. The empirical analysis does not sufficiently explain the performance differences, and the evaluation protocol leaves robustness and cost questions open.

The gain over GEPA is relatively small on AMG and DeceptionDecoded, while the paper reports means over three seeds without paired confidence intervals or significance tests for these comparisons. The larger MMFakeBench gain is encouraging, but the run-to-run variability also warrants a paired analysis. Further, because the same validation set is repeatedly used to diagnose errors and select candidates throughout evolution, the final prompt may adapt to idiosyncrasies of that set. The paper does not adequately analyze this risk or show which error categories are corrected by the feedback.

The compute comparison also matches call counts rather than input/output tokens, monetary cost, or wall-clock time. Calls used for multimodal inference, diagnosis, and candidate generation can have very different costs. Finally, the multi-backbone study is limited to AMG and a subset of baselines, so it supports preliminary robustness rather than broad cross-dataset generality. Reporting paired per-seed results, error-category transitions, a validation-to-test generalization analysis, and token/cost measurements would make the evidence more convincing.

Questions for authors
  1. Novelty and mechanism: What specifically distinguishes the proposed structured diagnosis from the natural-language reflection used by GEPA or PromptAgent? Do the authors have examples or existing analyses showing that diagnoses are consolidated into stable criteria, rather than remaining case-specific suggestions?
  2. Robustness and validation reuse: Can the authors report paired per-seed comparisons or confidence intervals against GEPA, especially on AMG and DeceptionDecoded? Given that the validation set is repeatedly used for diagnosis and selection, is there evidence that the gains are not primarily due to adaptive overfitting to that set?
  3. Practical cost and error analysis: Can the authors provide token usage and runtime, in addition to call counts, and identify which error types are corrected by the diagnostic feedback? This would clarify both the source of the gains and the actual cost of obtaining them.
4 comments
AuthorsOpenReview ↗

Request for Clarification Regarding Potential Review–Manuscript Mismatch

Dear Reviewer xRgj,

We sincerely thank you for your time and feedback. However, we noticed that several concerns raised in your review refer to concepts that are not discussed in our manuscript, including observability margins, linear transports, external anchors, and RRF recovery. The review also references Appendices Q.4 and Q.5, neither of which exists in our paper.

Our submission, GuidedEvolve: Feedback-Guided Prompt Evolution for Discovering Implicit Decision Criteria, focuses on structured diagnostic feedback and evolutionary prompt optimization, as described in Sections 3.1–3.3.

Given these discrepancies, we wonder whether some of the comments may have been intended for a different submission. We would greatly appreciate it if you could kindly verify whether your review corresponds to our manuscript.

Thank you for your time and consideration.

Reviewer_xRgjOpenReview ↗

Reponse to authors

Dear Authors,

I checked my original review doc, this is indeed a mistake. I have updated my review.

Thanks.

AuthorsOpenReview ↗

Response to Reviewer xRgj (Part 1)

Q1: Novelty and mechanism.

R1: We thank the reviewer for the suggestions. GuidedEvolve integrates structured case-level diagnosis, feedback accumulation, and evolutionary search through a nested inner–outer loop framework. To validate these components, we conduct two additional ablations: Free-form Feedback, replacing structured diagnosis with unrestricted feedback, and w/o Feedback Accumulation, using feedback from only one case per update.

MethodMMFakeBench (F1/Acc./Prec./Rec.)AMG (F1/Acc./Prec./Rec.)
Free-form Feedback47.12 / 51.91 / 52.38 / 46.6826.43 / 40.31 / 31.81 / 31.04
w/o Feedback Accumulation51.54 / 54.60 / 54.85 / 53.4823.57 / 42.10 / 28.10 / 26.21
GuidedEvolve (Full)58.15 / 59.58 / 61.66 / 57.8426.94 / 48.12 / 29.85 / 30.17

GuidedEvolve improves F1 over Free-form Feedback by 11.03/0.51 points and over w/o Feedback Accumulation by 6.61/3.37 points on MMFakeBench/AMG, supporting the contributions of structured diagnosis and multi-case feedback accumulation.

Representative Case. For MemeMind_train_002986 (“DONALD SHRUMP”), the original prompt incorrectly predicted harmful (1) instead of harmless (0). Diagnosis identified insufficient consideration of contextual humor and targeting intent, leading to the following guidance:

“Before labeling as harmful, verify whether the humor is neutralized by context (e.g., public figure name + pun) and whether it avoids targeting vulnerable groups or sensitive topics.”

The revised prompt also corrected another example outside the diagnostic batch:

ExampleGround TruthBefore RevisionAfter Revision
Chuck Norris memeHarmless (0)Harmful (1) ✗Harmless (0) ✓

This illustrates how case-level diagnosis can yield reusable decision criteria beyond the original diagnostic example.


Q2: Robustness and validation reuse.

R2: We provide additional analyses of statistical reliability, optimization sensitivity, and disjoint validation to assess robustness and potential validation overfitting.

(1) Paired Statistical Reliability. We compare GuidedEvolve against GEPA across three matched seeds, with 10,000 paired bootstrap resamples over aligned test predictions.

DatasetSeed 42Seed 43Seed 44Mean ΔF1SDBootstrap 95% CI
MMFakeBench0.884.608.744.743.93[4.02, 5.46]
AMG−0.01−0.292.820.841.72[−1.07, 2.69]
DeceptionDecoded−1.330.843.190.902.26[−0.69, 2.48]

GuidedEvolve achieves positive mean gains on all benchmarks, outperforming GEPA across all three seeds on MMFakeBench, whose prediction-level bootstrap interval is strictly positive. AMG and DeceptionDecoded show smaller, less consistent gains, with intervals including zero. Across-seed variability and prediction-level uncertainty are distinct, and additional seeds remain future work.

(2) Optimization Sensitivity Analysis. We evaluate validation/held-out test F1 (%) at different optimization iterations.

IterationsMMFake Val/TestAMG Val/TestDeception Val/Test
036.50 / 37.0221.18 / 23.0967.78 / 66.07
1051.79 / 39.8127.13 / 23.7870.66 / 68.66
2053.61 / 38.4629.05 / 24.3872.55 / 70.93
4060.93 / 48.3931.27 / 24.3974.20 / 72.56
6062.71 / 47.4832.41 / 25.4074.75 / 73.07
8063.91 / 54.4633.24 / 25.8375.61 / 73.88
10064.21 / 57.8133.60 / 26.9575.87 / 74.98

From iteration 0 to 100, held-out test F1 increases by 20.79, 3.86, and 8.91 points, respectively. Despite intermediate fluctuations and validation–test gaps, we observe no sustained deterioration in test performance within the evaluated optimization budget.

(3) Disjoint Validation Evaluation. We separate diagnostic feedback generation and candidate evaluation into non-overlapping validation subsets.

MethodMMFake F1AMG F1Deception F1
AlphaEvolve51.7424.1470.53
GuidedEvolve (Original)58.1526.9475.02
GuidedEvolve (Disjoint)52.3325.1274.02

Despite reduced performance relative to the original setting, **GuidedEvolve (Disjoint) still outperforms AlphaEvolve. Together, these results provide further evidence of GuidedEvolve's generalization capability and effectiveness under different validation protocols.

AuthorsOpenReview ↗

Response to Reviewer xRgj (Part 2)

Q3: Validation-set Overfitting and Generalization.

R3: We thank the reviewer for raising this concern. To assess potential overfitting from repeated validation reuse, we conduct two additional experiments: (1) optimization sensitivity analysis and (2) disjoint validation evaluation.

(1) Optimization Sensitivity Analysis. We evaluate the best prompts obtained at different optimization iterations on both validation and held-out test sets. The table below reports mean F1 scores (%).

IterationsMMFake Val/TestAMG Val/TestDeception Val/Test
036.50 / 37.0221.18 / 23.0967.78 / 66.07
1051.79 / 39.8127.13 / 23.7870.66 / 68.66
2053.61 / 38.4629.05 / 24.3872.55 / 70.93
4060.93 / 48.3931.27 / 24.3974.20 / 72.56
6062.71 / 47.4832.41 / 25.4074.75 / 73.07
8063.91 / 54.4633.24 / 25.8375.61 / 73.88
10064.21 / 57.8133.60 / 26.9575.87 / 74.98

Despite fluctuations at intermediate iterations, test F1 improves from iteration 0 to 100 across all three benchmarks, with gains of 20.79, 3.86, and 8.91 points, respectively. Notably, validation improvements are accompanied by overall test improvements rather than sustained test-performance degradation. These results suggest that repeated optimization does not cause progressive deterioration in held-out performance within the evaluated budget, although some validation–test gaps remain.

(2) Disjoint Validation Evaluation. We further separate the validation data into two non-overlapping subsets, using one for diagnostic feedback generation and the other for prompt evaluation and selection.

MethodMMFake F1AMG F1Deception F1
AlphaEvolve51.7424.1470.53
GuidedEvolve (Original)58.1526.9475.02
GuidedEvolve (Disjoint)52.3325.1274.02

Under disjoint validation, GuidedEvolve retains improvements over AlphaEvolve of 0.59, 0.98, and 3.49 F1 points, respectively. Although performance decreases relative to the original setting, these results indicate that the benefits of diagnostic feedback are not entirely dependent on using overlapping validation examples for diagnosis and candidate selection.

Overall, the two experiments provide complementary evidence: optimization sensitivity analysis demonstrates consistent overall improvements in held-out test performance across increasing optimization iterations, while disjoint validation preserves improvements over AlphaEvolve across all three benchmarks. These results provide complementary evidence for the effectiveness and robustness of GuidedEvolve across multiple benchmarks and under different validation protocols.

Reviewer_WEfcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GuidedEvolve as a framework for prompt optimization for multi-modal misinformation detection, where the core idea is to use feedback based guidance based on an inner and outer loop evolution. Experimented on Qwen3-VL-8B, it shows higher results on MMFakeBench, AMG and DecptionDecoded benchmarks compared to previous methods. The improvement is further validated across different models, as well as efficiency and ablation of each module proposed.

Critical strengths
  1. thorough ablation has been done across the modules themselves proposed and the token efficiency of the proposed method (yet some still missing, see the weaknesses part), and the proposed method achieved higher benchmark score than previous methods.
  2. the proposed methods also works across different VLMs.
  3. Overall a complete work
Critical weaknesses

First issue is with the claim of the novelty of feedback-based analysis and suggestion. In previous work like Gepa* also includes such detailed analysis, so the novelty and detailed difference needs to be further explained.

The second issue is that I find the work is missing some critical details, for example:

  1. the analyst used for comparing predictions as well as for generating diagnostic suggestions
  2. structure and further details of "structured diagnostic suggestion" (L218)
  3. the detail of how the suggestions are applied The missing details make it hard to dive into this methodology and how it exactly worked. These could lead to suspicion on many ways of how the better benchmark score is achieved, e.g.: it could be that strong analyst model can naturally produce better prompts.

Last issue is also related to the details, there are no exact prompt/suggestions case example shown, make it hard to understand how the proposed suggestion-related method improve the prompts.

*Gepa: reflective prompt evolution can outperform reinforcement learning

Questions for authors

Please considering provide more details as stated in critical weaknesses part, I will hold my rating before that.

1 comment
AuthorsOpenReview ↗

Response to Reviewer WEfc

Q1: Novelty of Feedback-Based Optimization and Distinction from Existing Methods.

R1: We thank the reviewer for the constructive feedback. GuidedEvolve distinguishes itself from existing feedback-based optimization methods through a nested inner–outer loop framework that integrates structured case-level diagnosis, feedback accumulation, and evolutionary search. The inner loop accumulates fine-grained diagnostic feedback across multiple cases, while the outer loop leverages this feedback for dual-candidate generation and multi-island evolution.

Furthermore, GuidedEvolve achieves the highest mean F1 scores across all three evaluated benchmarks, outperforming recent prompt optimization baselines, including AlphaEvolve, ShinkaEvolve, ThetaEvolve, and GEPA. These results, together with our component ablations, provide empirical support for the effectiveness of the proposed framework. We will further clarify these methodological distinctions and contributions in the revised manuscript.


Q2: Clarification of Diagnostic Feedback and Implementation Details.

R2: We thank the reviewer for the comments. We would like to clarify the following points.

(1) Analyst Model and Fairness. We use Qwen3-VL-8B for diagnostic analysis, candidate generation, and task inference. The same backbone is used across all baselines, including existing feedback-based methods such as GEPA and PromptAgent. Moreover, computational budgets are closely matched across methods, with differences in total LLM calls within 2.4%. Therefore, the improvements cannot simply be attributed to a stronger Analyst model or greater computational resources.

(2) Structured Diagnostic Suggestions. As described in Section 3.2 (Lines 218–227), each diagnostic suggestion consists of four fields: Error Type, identifying the failure category; Missing Factors, identifying overlooked evidence; Confusion Reason, explaining the incorrect judgment; and Revision Suggestion, providing actionable guidance for prompt refinement.

(3) Feedback Integration. As detailed in Section 3.3 and Algorithm 1, diagnostic suggestions are accumulated across sampled cases in the inner loop and incorporated into guided candidate generation in the outer loop. The guided candidate is evaluated alongside a standard evolutionary candidate, and the better-performing candidate is retained for subsequent evolution.

We believe these details clarify how diagnostic feedback contributes to prompt optimization. We will further improve the methodological presentation and release the complete implementation, including diagnostic and mutation prompts, upon acceptance.


Q3: Concrete Example of Feedback-Guided Prompt Revision.

R3: Thank you for the suggestion. We provide a concrete case study to illustrate how GuidedEvolve translates diagnostic feedback into actionable decision guidance and how the revised prompt improves predictions beyond the diagnostic examples.

(1) Diagnostic Feedback and Prompt Revision. For MemeMind_train_002986 (“DONALD SHRUMP”), the original prompt incorrectly classified the meme as harmful (1), whereas the ground-truth label was harmless (0). The four-field diagnosis identified a failure to distinguish contextual humor from harmful targeting and suggested examining the humor's context, target, and intent before making a prediction.

Based on this feedback, GuidedEvolve incorporated the following decision guidance into the revised prompt:

“Before labeling as harmful, verify whether the humor is neutralized by context (e.g., public figure name + pun) and whether it avoids targeting vulnerable groups or sensitive topics.”

(2) Effect on an Additional Example. To illustrate the effect of this revision, we examine MemeMind_train_012174 (Chuck Norris meme), which was not included in the diagnostic batch. The original prompt incorrectly predicted harmful (1), while the revised prompt correctly predicted harmless (0).

ExampleGround TruthBefore RevisionAfter Revision
Chuck Norris memeHarmless (0)Harmful (1) ✗Harmless (0) ✓

This example illustrates how GuidedEvolve converts a specific diagnostic failure into a reusable decision criterion, rather than merely correcting an individual prediction. The resulting guidance helps the model distinguish benign contextual humor from genuinely harmful targeting, improving its decision on an additional example without including that example in the diagnostic feedback.

Reviewer_FeDSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GuidedEvolve, a feedback-guided prompt evolution framework for multimodal misinformation detection. Instead of relying exclusively on aggregate fitness scores, the method uses an Analyst VLM to diagnose individual validation errors, identify missing evidence and reasoning factors, and generate actionable revision suggestions. These suggestions guide a candidate prompt alongside a standard evolutionary candidate, with selection based on validation performance. Experiments on MMFakeBench, AMG, and DeceptionDecoded report improvements over existing prompt optimization methods under approximately matched LLM-call budgets. Ablation studies suggest that both the guided candidate branch and diagnostic feedback contribute to the reported performance gains.

Critical strengths
  1. The paper addresses a relevant limitation of scalar-feedback-driven prompt optimization: aggregate metrics reveal whether a prompt performs well but provide limited information about why particular examples fail. The proposed case-level diagnosis and feedback-guided evolution offer a straightforward and potentially useful mechanism for directing prompt refinement toward specific error patterns.

  2. The experimental evaluation includes three misinformation benchmarks, several prompt optimization baselines, approximately matched LLM-call budgets, and component ablations. GuidedEvolve achieves its strongest improvement on MMFakeBench, and removing either the guided candidate branch or diagnostic feedback substantially reduces performance. These results provide preliminary evidence that the proposed feedback mechanism contributes beyond simply generating additional prompt candidates.

Critical weaknesses
  1. The paper does not sufficiently establish that the diagnostic feedback captures reliable and transferable decision criteria, rather than benchmark-specific patterns or noisy explanations generated by the Analyst VLM. The ablations demonstrate the utility of the overall feedback mechanism, but they do not directly assess the accuracy, consistency, or generalizability of the generated diagnoses. Moreover, prompt diagnosis and candidate selection repeatedly use the same validation set. Although separate test sets are used for final evaluation, the experiments do not establish whether the discovered decision strategies transfer to previously unseen datasets without re-optimization. This limits the broader claim that GuidedEvolve discovers reusable decision principles.

  2. The statistical evidence for the claimed consistent improvements is not fully convincing. The repeated-run results use only three random seeds, and no statistical significance tests are reported. In particular, the F1 improvements over GEPA on AMG and DeceptionDecoded are only 0.84 and 0.90 percentage points, respectively, while the reported run-to-run variability is substantial relative to these gains. For example, GuidedEvolve obtains 26.94 ± 1.25 F1 on AMG compared with 26.10 ± 2.17 for GEPA. The results are encouraging, but the current evidence does not establish that the smaller gains are robust across optimization runs. Additional statistical analysis is needed to support the broader performance claims.

Questions for authors
  1. How do the authors validate the correctness and consistency of the Analyst VLM's error diagnoses? Can the proposed feedback be evaluated against independently annotated error categories or tested for transfer to a different benchmark without re-optimizing the prompt?

  2. Can the authors provide paired statistical tests or additional independent runs to establish whether the improvements over GEPA on AMG and DeceptionDecoded are statistically reliable?

  3. How sensitive is GuidedEvolve to repeated optimization on the same validation set, and would a nested validation or cross-dataset evaluation protocol preserve the reported improvements?

2 comments
AuthorsOpenReview ↗

Response to Reviewer FeDS (Part 1)

Q1: Transferability of Learned Decision Criteria.

R1: We thank the reviewer for raising this point. GuidedEvolve demonstrates that structured diagnostic feedback can produce actionable decision criteria that improve predictions beyond the individual examples used for diagnosis.

(1) Diagnosis-to-Decision Case Study. For MemeMind_train_002986 (“DONALD SHRUMP”), the original prompt incorrectly predicted harmful (1) instead of harmless (0). The diagnosis identified a failure to distinguish contextual humor from harmful targeting and suggested considering the humor's context, target, and intent. The revised prompt incorporated the following guidance:

“Before labeling as harmful, verify whether the humor is neutralized by context (e.g., public figure name + pun) and whether it avoids targeting vulnerable groups or sensitive topics.”

This guidance also helped correct the prediction for another example, MemeMind_train_012174 (Chuck Norris meme), which was not included in the diagnostic batch:

ExampleGround TruthBefore RevisionAfter Revision
Chuck Norris memeHarmless (0)Harmful (1) ✗Harmless (0) ✓

This case illustrates how a decision criterion derived from one diagnostic example can improve predictions on another example within the same dataset, rather than merely correcting the original error.

(2) Scope of Transferability. Our current experiments demonstrate the effectiveness of the learned decision criteria on held-out test data across multiple benchmarks and model backbones. While these evaluations focus on performance within each experimental setting, cross-backbone and cross-dataset transfer without reoptimization represents a complementary direction for further investigation. We will clarify the scope of the current findings in the revised manuscript and highlight broader transferability evaluation as future work.


Q2: Statistical Significance and Reliability of Performance Improvements.

R2: To further assess the statistical reliability of GuidedEvolve's improvements over GEPA, we analyze three archived runs per method and conduct additional statistical tests. Specifically, we report paired F1 differences across runs, two-sided paired t-tests, and paired bootstrap confidence intervals based on test predictions.

DatasetMean ΔF1SD of ΔF1Positive Run Pairsp-value95% Bootstrap CI
MMFakeBench4.743.933/30.172[4.02, 5.46]
AMG0.841.721/30.487[−1.07, 2.69]
DeceptionDecoded0.902.262/30.563[−0.69, 2.48]

ΔF1 denotes GuidedEvolve minus GEPA in F1 percentage points. SD is the sample standard deviation across three provisionally paired runs. The p-values are from two-sided paired t-tests (n = 3), while the 95% confidence intervals are obtained from 10,000 paired prediction-level bootstrap iterations.

The results show that GuidedEvolve achieves higher mean F1 across all three benchmarks, despite some run-to-run variability. Notably, on MMFakeBench, GuidedEvolve outperforms GEPA in all three seed-matched comparisons, and the prediction-level bootstrap confidence interval remains strictly positive, providing additional evidence of improvement conditional on the evaluated prompts. For AMG and DeceptionDecoded, the mean improvements remain positive, although the confidence intervals include zero. Given the substantial computational time required for each optimization run, we currently report results from three random seeds and plan to extend the evaluation to additional seeds in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer FeDS (Part 2)

Q3: Validation-set Overfitting and Generalization.

R3: We thank the reviewer for raising this concern. To assess potential overfitting from repeated validation reuse, we conduct two additional experiments: (1) optimization sensitivity analysis and (2) disjoint validation evaluation.

(1) Optimization Sensitivity Analysis. We evaluate the best prompts obtained at different optimization iterations on both validation and held-out test sets. The table below reports mean F1 scores (%).

IterationsMMFake Val/TestAMG Val/TestDeception Val/Test
036.50 / 37.0221.18 / 23.0967.78 / 66.07
1051.79 / 39.8127.13 / 23.7870.66 / 68.66
2053.61 / 38.4629.05 / 24.3872.55 / 70.93
4060.93 / 48.3931.27 / 24.3974.20 / 72.56
6062.71 / 47.4832.41 / 25.4074.75 / 73.07
8063.91 / 54.4633.24 / 25.8375.61 / 73.88
10064.21 / 57.8133.60 / 26.9575.87 / 74.98

Despite fluctuations at intermediate iterations, test F1 improves from iteration 0 to 100 across all three benchmarks, with gains of 20.79, 3.86, and 8.91 points, respectively. Notably, validation improvements are accompanied by overall test improvements rather than sustained test-performance degradation. These results suggest that repeated optimization does not cause progressive deterioration in held-out performance within the evaluated budget, although some validation–test gaps remain.

(2) Disjoint Validation Evaluation. We further separate the validation data into two non-overlapping subsets, using one for diagnostic feedback generation and the other for prompt evaluation and selection.

MethodMMFake F1AMG F1Deception F1
AlphaEvolve51.7424.1470.53
GuidedEvolve (Original)58.1526.9475.02
GuidedEvolve (Disjoint)52.3325.1274.02

Under disjoint validation, GuidedEvolve retains improvements over AlphaEvolve of 0.59, 0.98, and 3.49 F1 points, respectively. Although performance decreases relative to the original setting, these results indicate that the benefits of diagnostic feedback are not entirely dependent on using overlapping validation examples for diagnosis and candidate selection.

Overall, the two experiments provide complementary evidence: optimization sensitivity analysis demonstrates consistent overall improvements in held-out test performance across increasing optimization iterations, while disjoint validation preserves improvements over AlphaEvolve across all three benchmarks. These results provide complementary evidence for the effectiveness and robustness of GuidedEvolve across multiple benchmarks and under different validation protocols.

Reviewer_ppBhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GuidedEvolve combines multi-island prompt evolution with case-level diagnostic feedback. Within each outer iteration, an Analyst VLM compares predictions from a fixed parent prompt against labeled validation cases and accumulates four fields: Error Type, Missing Factors, Confusion Reason, and Revision Suggestion. The generator produces both a standard evolutionary candidate and a feedback-guided candidate; validation fitness determines which enters the population, followed by island migration. The feedback collection is reset at the next outer iteration, and the final prompt is selected by validation fitness. This is labeled-validation-driven prompt optimization, not unsupervised policy discovery. On MMFakeBench, AMG, and the DeceptionDecoded source task, the method reports higher mean F1 than the compared optimizers under approximately matched total LLM-call budgets. Component ablations and AMG evaluations across four VLMs provide useful supporting evidence. The central question is whether these improvements establish a distinct contribution beyond existing feedback-based prompt optimization and substantiate the stronger claim of discovering interpretable, transferable decision criteria.

Critical strengths

S1. A practical and understandable optimization design The four-field diagnosis connects individual model behavior to actionable prompt revisions, while the standard candidate preserves an exploration path when diagnostic guidance is unhelpful. Figure 1 and Algorithm 1 make the nested loops and candidate selection easy to follow. The approach adapts model behavior without updating VLM parameters, which is attractive when weight-level training is unavailable.

S2. Positive mean results across three benchmarks Tables 1 and 4 report the highest mean F1 among the compared methods on all three datasets. The strongest competing baseline is GEPA,The MMFakeBench gain is particularly encouraging. These are F1 percentage-point differences, not relative percentage improvements or established significance results. The benchmarks include both binary and multiclass settings, making the evaluation more informative than a single-task demonstration.

S3. Ablations support the usefulness of diagnostic guidance Table 3 reports lower F1 when the guided candidate is removed (51.74, 24.14, 70.53) or when diagnostic feedback is bypassed (45.61, 23.92, 68.75), compared with Full (58.15, 26.94, 75.02). This supports the usefulness of the feedback-guided branch within the implemented system and argues against attributing its benefit simply to generating another unguided candidate. It does not yet isolate the four-field structure from other forms of natural-language feedback, as discussed in W1.

S4. Attention to budgets, repeated runs, and evaluation boundaries Table 2 reports total LLM-call deviations within 2.4%, and Appendix A provides three-run means and standard deviations for seeds 42, 43, and 44. The paper explicitly separates validation-based optimization from final test evaluation; the DeceptionDecoded validation and test sets each contain 1,000 non-overlapping examples. Figure 2 also reports gains over Standard prompting and AlphaEvolve across four VLMs on AMG. These are meaningful checks, even though they do not establish equal token costs or transfer of a fixed learned policy.

Critical weaknesses

W1. The contribution beyond existing feedback-based optimization is insufficiently isolated (Major) GEPA already uses natural-language reflection, and PromptAgent uses error feedback; the paper itself acknowledges both. The novelty is therefore the specific combination of four-field diagnosis, within-iteration accumulation, and dual-candidate island evolution—not the introduction of feedback into scalar-only search. Table 3 supports diagnosis over no diagnosis but does not compare structured diagnosis with free-form feedback under the same searcher, cases, and budget. Such a targeted control would establish what the proposed structure adds beyond established feedback optimization, without requiring an unrelated new baseline or a more complex method.

W2. Interpretable and transferable decision criteria are not directly validated (Major) The manuscript describes diagnostic fields but provides little concrete evidence tracing a case diagnosis into an extracted rule, a revised prompt, and corrected behavior on unseen cases. Better benchmark scores establish useful optimization, not necessarily faithful error explanations or reusable criteria. Figure 2 shows effectiveness across AMG backbones, not transfer of a fixed prompt or rule set without reoptimization. A complete diagnosis-to-rule example and a targeted rule-removal or held-out transfer test would strengthen the claim. Alternatively, the authors could narrow the contribution to structured-feedback-guided prompt optimization.

W3. The reliability of the smaller gains needs clearer characterization (Moderate) The GEPA improvements on AMG and DeceptionDecoded are 0.84 and 0.90 F1 points, while the reported three-run standard deviations are non-negligible. MMFakeBench shows a larger mean gain, but GuidedEvolve also has higher run-to-run variation (4.29 versus 1.47). These summaries establish neither significance nor equivalence, and higher means do not show superiority in every seed. Please provide seed-matched results and, where feasible, paired prediction-level uncertainty, while distinguishing search variability from uncertainty across examples. This qualifies the strength of the evidence rather than invalidating the positive means.

W4. Reproducibility and cost accounting remain incomplete (Moderate) The paper specifies the default inference VLM, decoding settings, splits, and total call counts, but leaves important role-specific and search details unclear: Analyst/generator identities, diagnostic and mutation prompts, island settings, batch sizes, and feedback aggregation. The promised release would help, but essential settings should be specified in the submission. Matching calls within 2.4% does not establish matched tokens, latency, or monetary cost when diagnosis and evaluation differ. A per-role configuration and cost breakdown would clarify reproducibility and practical efficiency; Figure 3 should also use cumulative calls or tokens before supporting cost-efficiency claims.

Questions for authors

Q1. Regarding W1: With the evolutionary searcher, sampled cases, candidate count, and budget fixed, how does four-field feedback compare with free-form error analysis? Can the authors identify the incremental benefit of feedback accumulation separately from candidate selection?

Q2. Regarding W2: Could the authors show a complete diagnosis → decision rule → prompt revision → unseen-case outcome? Do any learned prompts or rules transfer to a different backbone or dataset without reoptimization, or should “transferable criteria” be treated as a hypothesis rather than an established result?

Q3. Regarding W3: What are the per-seed differences against GEPA, especially on AMG and DeceptionDecoded? Could uncertainty be reported using paired runs or predictions, with clear separation between search stochasticity and test-sample uncertainty?

Q4. Regarding W4: Which models perform diagnosis, candidate generation, and evaluation, and what are their full prompts and search configurations? How are calls divided among these roles, and do conclusions remain similar when measured by tokens or runtime rather than call count?

2 comments
AuthorsOpenReview ↗

Response to Reviewer ppBh (Part 1)

Q1: Additional ablation studies and clarification of novelty claims.

R1: We thank the reviewer for the suggestions. Our contribution lies in integrating structured case-level diagnosis and feedback accumulation with evolutionary search through a nested inner–outer loop framework, enabling systematic identification and refinement of actionable decision criteria. To further validate these contributions, we conduct two additional ablations under the same search configuration: (1) Free-form Feedback, which replaces structured diagnosis with unrestricted natural-language feedback while retaining multi-case aggregation, and (2) w/o Feedback Accumulation, which retains structured diagnosis but uses feedback from only one sampled case per prompt update.

MethodMMFakeBench (F1/Acc./Prec./Rec.)AMG (F1/Acc./Prec./Rec.)
Free-form Feedback47.12 / 51.91 / 52.38 / 46.6826.43 / 40.31 / 31.81 / 31.04
w/o Feedback Accumulation51.54 / 54.60 / 54.85 / 53.4823.57 / 42.10 / 28.10 / 26.21
GuidedEvolve (Full)58.15 / 59.58 / 61.66 / 57.8426.94 / 48.12 / 29.85 / 30.17

The results provide preliminary evidence supporting both structured diagnosis and multi-case feedback accumulation. Compared with Free-form Feedback, GuidedEvolve improves F1 by 11.03 and 0.51 points on MMFakeBench and AMG, respectively. Removing feedback accumulation leads to larger F1 reductions of 6.61 and 3.37 points, respectively. These findings suggest that structuring case-level feedback and accumulating diagnostic information can provide more effective guidance for prompt evolution than unrestricted feedback or isolated case-level revisions.

We will also revise the manuscript to clarify our novelty claims and distinguish GuidedEvolve from GEPA and PromptAgent.


Q2: Validation and transferability of discovered decision criteria.

R2: We thank the reviewer for raising this point. GuidedEvolve demonstrates that structured diagnostic feedback can produce actionable decision criteria that improve predictions beyond the individual examples used for diagnosis.

(1) Diagnosis-to-decision case study.

For MemeMind_train_002986 (“DONALD SHRUMP”), the original prompt incorrectly predicted harmful (1) instead of harmless (0). The diagnosis identified a failure to distinguish contextual humor from harmful targeting and suggested considering the humor's context, target, and intent. The revised prompt incorporated the following guidance:

“Before labeling as harmful, verify whether the humor is neutralized by context (e.g., public figure name + pun) and whether it avoids targeting vulnerable groups or sensitive topics.”

This guidance also helped correct the prediction for another example, MemeMind_train_012174 (Chuck Norris meme), which was not included in the diagnostic batch:

ExampleGround TruthBefore RevisionAfter Revision
Chuck Norris memeHarmless (0)Harmful (1) ✗Harmless (0) ✓

This case illustrates how a decision criterion derived from one diagnostic example can improve predictions on another example within the same dataset, rather than merely correcting the original error.

(2) Scope of transferability.

Our current experiments demonstrate the effectiveness of the learned decision criteria on held-out test data across multiple benchmarks and model backbones. While these evaluations focus on performance within each experimental setting, cross-backbone and cross-dataset transfer without reoptimization represents a complementary direction for further investigation. We will clarify the scope of the current findings in the revised manuscript and highlight broader transferability evaluation as future work.

AuthorsOpenReview ↗

Response to Reviewer ppBh (Part 2)

Q3: Per-seed performance and statistical reliability.

R3: We thank the reviewer for the suggestion. We provide seed-matched comparisons and paired bootstrap analyses to distinguish search variability from test-example uncertainty.

(1) Seed-matched performance comparison.

The table reports macro-F1 differences (GuidedEvolve − GEPA, percentage points) across three seeds. GuidedEvolve achieves higher mean F1 on all benchmarks, outperforming GEPA across all seeds on MMFakeBench, while AMG and DeceptionDecoded show smaller, more variable gains.

DatasetSeed 42Seed 43Seed 44Mean ΔF1
MMFakeBench0.884.608.744.74
AMG−0.01−0.292.820.84
DeceptionDecoded−1.330.843.190.90

(2) Statistical reliability analysis.

We report across-seed standard deviations and 95% confidence intervals from 10,000 paired bootstrap resamples of aligned test predictions.

DatasetMean ΔF1SD of ΔF1Bootstrap 95% CI
MMFakeBench4.743.93[4.02, 5.46]
AMG0.841.72[−1.07, 2.69]
DeceptionDecoded0.902.26[−0.69, 2.48]

MMFakeBench shows a positive bootstrap interval, whereas those for AMG and DeceptionDecoded include zero. Across-seed SD reflects search variability, while bootstrap intervals capture test-example uncertainty conditional on the fitted prompts, without accounting for repeated optimization.


Q4: Reproducibility and computational cost analysis.

R4: We thank the reviewer for the comments. We clarify implementation details, computational allocation, and convergence under matched call and token budgets.

(1) Reproducibility and search configuration.

All roles (diagnosis, generation, evaluation) use Qwen3-VL-8B. Generation uses temperature/top-p = 0.7/0.95; diagnosis and evaluation use 0/1. We use seeds 42–44, 100 iterations, 3 islands, population/archive capacities of 50/20, elite ratio 0.2, exploitation probability 0.7, concurrency 4, and diagnostic batch size 10. These settings will be included in the revision, with code released upon acceptance.

(2) Role-wise computational breakdown.

Aggregated nominal LLM calls across three benchmarks:

OperationLLM CallsShare
Diagnostic Predictions3,0000.59%
Analyst Diagnosis3,0000.59%
Standard Candidate Generation3000.06%
Guided Candidate Generation3000.06%
Fitness Evaluation505,91798.71%
Total512,517100%

Fitness evaluation accounts for 98.71% of calls. GuidedEvolve and AlphaEvolve have closely matched nominal call budgets, differing by only 0.19% overall.

(3) Computational efficiency under calls and tokens.

We compare best-so-far validation F1 against cumulative LLM calls and estimated text tokens, using identical absolute budget thresholds for both methods within each benchmark.

Table R1: Call-based convergence (validation F1, %)

DatasetCallsAlphaEvolveGuidedEvolve
MMFakeBench40K53.0247.42
80K54.9755.08
120K54.9758.97
160K54.9759.31
AMG20K25.6729.54
40K25.6732.66
60K26.6433.76
80K26.6435.02
DeceptionDecoded40K71.8672.37
80K73.8274.26
120K73.8275.10
160K73.8275.86

Table R2: Token-based convergence (validation F1, %)

DatasetTokensAlphaEvolveGuidedEvolve
MMFakeBench10M49.9045.88
20M53.0250.67
30M54.9754.77
40M54.9756.04
AMG10M25.6726.86
20M25.6729.54
30M25.6730.53
40M26.6432.66
DeceptionDecoded50M71.8671.07
100M73.8274.26
150M73.8275.10
200M73.8275.61

Note: Results show best-so-far validation F1 within each budget. Calls are reconstructed from nominal allocations; tokens are estimated from input/output text.

GuidedEvolve achieves higher F1 at most matched budgets, consistently outperforming AlphaEvolve on AMG, surpassing it after the initial token checkpoint on DeceptionDecoded, and overtaking it at approximately 80K calls or 40M tokens on MMFakeBench. These results support the effectiveness of structured diagnostic feedback under resource-aligned comparisons, beyond iteration-based convergence alone.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.