acceptodds
Under review as a conference paper at ICLR 2027

Improving scientific chart analysis in VLMs via autonomous evolution

Abstract

Understanding and operating with scientific charts is a standard approach in data analysis, and while humans are good at interpreting such charts, open vision-language models (VLMs) are known to perform poorly when it comes to question answering based on such data types. Motivated by an initial observation that many of their errors are due to inexact answering, we raise the following natural question: how much accuracy can we recover without changing the model at all? In this paper, we study this question by searching over inference programs and the code around a “frozen” version of the model. Using ideas from autonomous evolution (famously used in AlphaEvolve), we first start from a baseline inference program, and in each round we show an LLM-judge the code together with a few failures on training questions, and ask for an improving evolutionary move. As usual in autonomous evolution approaches, the candidate programs are kept in a small pool ranked by training accuracy, and the next edit is performed on top-scorers from this pool. We run this approach on three VLMs and four popular benchmarks on scientific charts, and we measure the fraction of descriptive questions whose answer exactly matches the ground truth. The accuracy improves in all experimental settings, and on average, it rises from 16.6% with the starting program to 41.3% with the evolved one, while on some cases, the largest single increase is 55.2 percentage points. As a side benefit, our evolved program is 3.8× faster than the starting program, because short answers are easier to grade and faster to generate. Our results demonstrate that a large part of the observed errors arise due to the inference pipeline, and that optimizing it can improve the “frozen” VLM without the need to retrain it.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
vision language models, chart question answering, LLM-guided program search, evolutionary code search, prompt optimization, exact match evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jX1BOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether chart question answering can be improved by optimizing the inference program around a frozen VLM, without retraining. Gemini 2.5 Flash uses failure examples to evolve prompts, question-type-specific hints, generation budgets, and output post-processing. Candidates undergo static validation, with parents selected by training accuracy and the final program selected by holdout accuracy. Experiments cover three VLMs and four chart benchmarks, reporting an increase in average descriptive exact-match accuracy from 16.6% to 41.3% across twelve combinations. A ChartQA case study also compares against a hand-tuned program and reports a 3.8× throughput improvement over the seed. The paper attributes the benefits partly to shorter, better-formatted answers.

Critical strengths
  1. Optimizing prompts, generation budgets, and answer extraction together is a practical approach that requires no model retraining. Searching over executable code allows these components to be adjusted jointly rather than tuning the prompt alone. The resulting inference program can be reused without running evolutionary search for each new question, making it potentially useful for improving existing VLM applications with limited training resources.

  2. Experiments cover twelve model–benchmark combinations across three VLMs and four datasets, all showing improvements over their respective seeds. This provides broader empirical coverage than a single-model or single-benchmark demonstration. The efficiency analysis also shows that higher exact-match accuracy can accompany shorter, faster generation, highlighting the practical importance of inference-pipeline configuration for both accuracy and deployment efficiency.

Critical weaknesses
  1. The additional value of evolutionary search is insufficiently established. The seed is explicitly described as having accuracy “low by design,” without formatting hints or post-processing. In the only stronger comparison, the hand-tuned program achieves 56.3% accuracy and 1.20 queries/s, versus 58.6% and 1.24 queries/s for best_speed. These differences are considerably smaller than the headline gains. There are no prompt-optimization comparisons or component ablations separating prompt changes, token budgets, and post-processing. A larger search space alone does not establish an advantage over simpler optimization methods. Moreover, Table 3 includes incorrect numerical predictions, so these examples cannot establish that formatting corrections account for most of the improvement.

  2. The evaluation does not clearly establish independent generalization. The holdout window is repeatedly used for program selection and therefore serves as a validation set. Table 1 evaluates the “full descriptive portion” without clearly stating whether search and selection examples are excluded; Table 4 uses 128 descriptive questions, matching the combined training and holdout sizes. The 26-question holdout is also small, and no variability across independent searches is reported. The relationship between the two evaluations also needs clarification: the seed’s descriptive EM is 0.5% on the full ChartQA descriptive portion in Table 1, versus 38.3% on the 128-question window in Table 4. Although the evaluation sets differ, the paper does not explain the window selection or whether other evaluation settings change. Finally, results under the official benchmark protocols would help establish whether the gains extend beyond answer-format compliance.

Questions for authors
  1. What are the search-training, selection-validation, and final-test splits and sample counts for each benchmark? Table 1 is computed on the "full descriptive portion" (described as "not only the holdout window"), while the 128 descriptive and 64 reasoning questions in Table 4 match the combined training and holdout sizes (102 + 26 and 51 + 13). Do these evaluations include examples used for search or program selection? Could you report results on a test set fully disjoint from both?

  2. The Qwen3-VL-2B/ChartQA seed scores 0.5% in Table 1 but 38.3% in Table 4. Do the seed code, prompts, answer normalization, or question sampling differ between these evaluations? If only the evaluated subset changes, how was the Table 4 subset selected?

  3. How much improvement remains over a simple baseline with a short-answer instruction, a fixed answer extractor, and a comparable generation budget? A comparison with a prompt-only optimizer using comparable numbers of optimizer calls and candidate evaluations would help establish the additional value of program-level search. Controlled ablations of prompt changes, token budgets, and post-processing would also clarify which components drive the gains.

  4. The qid examples in Sec. 3.1 ("highest y-tick", "number of series") resemble CharXiv's templated descriptive questions. How are the descriptive subsets and qid labels obtained for ChartQA, PlotQA, and EvoChartQA, and are they available at deployment time? Do any evolved per-qid rules encode benchmark-specific answer patterns? Separately, how is the answer normalized, and do the gains hold under each benchmark's official evaluation protocol?

  5. Table 3 shows incorrect values in the seed outputs, such as 58 instead of 60 and 28 instead of 38, while the evolved programs return the correct answers. These displayed differences are not explained by removing extra text alone. Since the seed outputs are abbreviated and the evolved answers may include post-processing, a worked example with the relevant code changes and full raw VLM outputs would help clarify whether the correction comes from a changed model response, answer extraction, or another post-processing step.

1 comment

We thank the reviewer for the valuable review. The requested held-out, official-metric comparisons are included, with all changes marked in blue.

Comment: What are the search, selection, and final-test splits? Are the Table 4 samples reused?

Response: In our observed and submittes study, yes. Search uses 102 descriptive questions, selection uses 26, and Table 4 scores their union of 128 (51 + 13 = 64 for reasoning). The holdout chooses the saved program, so it is a validation set, and we now write this and clarify explicitly. Table 1 is the full descriptive portion of each benchmark (seed / evolved):

ModelCharXivChartQAEvoChartQAPlotQA
Qwen3-VL-2B38.3 / 66.40.5 / 55.71.0 / 47.40.0 / 33.3
InternVL323.4 / 52.349.5 / 62.06.8 / 36.510.9 / 13.5
SmolVLM235.2 / 37.511.5 / 41.19.9 / 28.612.5 / 21.4

We also do a test on a 1,000-question subset of the official ChartQA test set (500 human-written and 500 automatically generated questions, one per distinct chart).

Comment: Why is the ChartQA seed 0.5% in Table 1 but 38.3% in Table 4?

Response: Thank you for catching this. The two cells are the same seed scored on two different sets, the full ChartQA descriptive portion (0.5%, Table 1) and the 128-question case-study window (38.3%, Table 4). The gap is not an improvement, and the paper now labels each cell by its set. The performance claim now comes applied on the shared held-out test, scored with relaxed accuracy, the recorded metric is desccribed as follows:

MethodQwen3-VL-2BQwen3-VL-32B
Manual wrapper59.0%88.0%
Prompt-only, matched budget61.0%89.0%
First automated edit60.2%89.2%
Evolved program66.5%91.5%

Comment: What remains beyond short-answer prompting, extraction, and token limits? Compare with prompt-only optimization at a matched budget.

Response: We now ran this comparison at the same candidate-evaluation budget, where the prompt-only arm may change only the prompt. The evolved program is 5.5 points above prompt-only optimization on the 2B model and 2.5 points above it on the 32B model (95% bootstrap CI [+0.9, +4.1]). The manual wrapper already includes a short-answer instruction, an extractor and a token limit, and the evolved program is 7.5 and 3.5 points above it (CI [+1.7, +5.3]). Restoring the manual prompt, extractor or token budget costs 4.2, 2.7 and 1.1 points.

Comment: Explain descriptive subsets, qid labels, normalization, and the official evaluation protocols.

Response: The descriptive subsets are the descriptive questions of each benchmark. qid is the question-type identifier as observed in Section 3.1. CharXiv supplies it with its questions, and on the other three benchmarks it is the type the question itself asks for, so we get it at deployment. The gold answer is never an input. Normalization is each benchmark's own. We now begin with the official metric, keep strict exact match as a diagnostic, and corrected our statement on CharXiv, whose released evaluator uses an LLM judge.

Comment: Table 3 changes incorrect numerical values. How can these examples support a formatting explanation?

Response: They cannot, and you are in-fact right to reject that reading.Thank you so much. The rows of Table 3 are:

QuestionGoldSeed returnsEvolved
Highest labeled y-tick?6058, inside a sentence60
Year of largest blue-green gap?2018a calculation at 20052018
% supporting a TikTok ban, Sept.3828%38
% with prior military service, 20231716%17

Removing the words around 58 does not produce 60, and removing the percent sign from 28% does not produce 38, so the returned value itself changed. We rewrote this paragraph to make it more clear. The experimental run separates the two mechanisms. Restoring the manual prompt costs 4.2 points, so the generation changes with the prompt, and restoring the manual extractor costs 2.7 points, so extraction contributes after the generation.

Comment: The evidence for accuracy, speed, and generalization is weaker than the headline suggests.

Response: We agree, and now write each claim with a comparison supporting it. On accuracy, the claim is the held-out table above, where the evolved program beats both a competent manual wrapper and matched prompt-only optimization on two model sizes. The first automated edit only matches prompt-only optimization, so the gain is not a single short-answer edit. On speed, best_speed is 3.8$\times$ faster than the seed and about 3.3% faster than the hand-tuned program. On reasoning exact match, which was never the fitness, the hand-tuned program leads (17.2% against 14.1%), and we report that as a loss.

We sincerely thank the reviewer for their efforts and constructive feedback, and are happy to clarify anything further.

Reviewer_D9nmOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes an LLM-driven evolutionary search framework that optimizes the Python inference wrapper around frozen VLMs to improve EM accuracy and inference speed on scientific chart QA benchmarks.

Critical strengths

The reviewer does not identify substantive strength from the paper.

Critical weaknesses
  1. Flawed Evaluation Method, Weak Baseline Comparisons
  • Using strict Exact Match here is not proper (lines 202–203 note that "correct values are marked incorrect if they are wrapped in sentences or extra units"). Grading an unconstrained prompt with strict Exact Match is unfair; if an LLM-as-a-judge were used, baseline performance would be much higher. These benchmarks officially use other grading methods to evaluate final answers, such as LLM-as-a-judge or relaxed numerical grading.
  • The simple human-written manual baseline is already strong, and the proposed method is essentially doing the same thing by adding output-formatting instructions so that "our method produce[s] short and concise answers" (line 155). In Figure 4, the evolved method performs basically the same as the simple human manual baseline, and no statistical significance tests are run in the paper. Furthermore, the paper only compares against the human manual prompt on Qwen3-VL-2B in ChartQA; this baseline should be compared across all 12 model–benchmark pairs in Table 1.
  • Based on the attached code and the paper's description, the Table 1 result is evaluated on the exact samples used during evolution (train + holdout), rather than on a separate unseen test set.
  1. Diminishing Returns Beyond Generation 1 and Missing Details for Reproducibility
  • From the provided code, the accuracy improvement comes mainly from the very first generation, while the remaining generations provide minimal or no improvements at all. This further suggests that simply instructing the model to output a short answer accounts for nearly 100% of both the accuracy jump and the speedup.
  • The authors should include the exact prompt strings, the program code at Generation 0 and Generation 25, and the human manual baseline prompts. Without these, the reviewer and readers cannot evaluate how much of the reported accuracy and throughput gains simply stem from adding a basic conciseness instruction and token cap.
  1. Missing Method and Experiment Details
  • It is unclear how qid is generated or where it comes from. It is also unclear whether qid contains information not present in the original questions, which could provide an unintended hint that helps the model produce the right output format for the Exact Match metric.
  • The paper states that the inference program p calls the model function f zero or more times, leaving it unclear how this affects overall speed and efficiency.
  • The paper does not specify which model sizes of InternVL3 and SmolVLM2 are used in the experiments.
Questions for authors

Please address the questions raised in the weakness section.

1 comment

We thank the reviewer for the clear and constructive review. The requested comparisons are included, with all changes marked in blue.

Comment: Strict exact match unfairly penalizes verbose but correct answers and does not reflect the official benchmark grading protocols.

Response: We thank the reviewer for pointing this out and we have corrected this. Strict exact match is now described as a diagnostic of the output format and not as our official protocol, and we corrected our description of CharXiv, it's released evaluator uses LLM grading. We added a test with the official ChartQA metric, relaxed accuracy, on 1,000-question subset of the official test set (500 human-written and 500 automatically generated questions, one per distinct chart), held out from search and from every selection decision. On Qwen3-VL-2B-Instruct:

MethodRelaxed accuracy
Manual wrapper59.0% (590/1,000)
Prompt-only, matched budget61.0% (610/1,000)
First automated edit60.2% (602/1,000)
Evolved program66.5% (665/1,000)

The gain is reported as 7.5 points over manual tuning and 5.5 points over prompt-only optimization.

Comment: The manual baseline is already strong. Compare it across all 12 pairs and test significance.

Response: We thank the reviewer for pointing this out and we now add clarifictaions. On this test, the evolved program answers 75 more questions than the manual wrapper and 55 more than the matched prompt search. On Qwen3-VL-32B-Instruct, on the same questions, it reaches 91.5% against 88.0% and 89.0%. The paired bootstrap 95% intervals are [+1.7, +5.3] and [+0.9, +4.1] points, and both exclude zero. We agree that the earlier Table 4 margin (three answers out of 128) was too small for the significance claim. Against the naive seed, strict exact match improves on all 12 pairs, from a mean of 16.6% to 41.3% (seed / evolved):

ModelCharXivChartQAEvoChartQAPlotQA
Qwen3-VL-2B38.3 / 66.40.5 / 55.71.0 / 47.40.0 / 33.3
InternVL323.4 / 52.349.5 / 62.06.8 / 36.510.9 / 13.5
SmolVLM235.2 / 37.511.5 / 41.19.9 / 28.612.5 / 21.4

Comment: The final evaluation includes the examples used during evolution and model selection.

Response: This is a fair point, and the new test removes that overlap. Its 1,000 questions take no part in mutation feedback, candidate selection, manual-prompt development or the choice of deployed program. The paper now calls the old holdout a validation set, and Table 1 is labeled as full-set numbers.

Comment: Almost all improvement appears in generation 1. Provide the exact prompts and programs.

Response: On the held-out test, generation 1 is not where the gain is. The first automated edit reaches 60.2% and the selected program 66.5%, so 6.3 points come after the first edit (+2.3 points on the 32B model, from 89.2% to 91.5%). Restoring the manual prompt or token budget costs 4.2, 2.7 and 1.1 points, so the final program uses all three. Generation 0 is a plain prompt, a generous token budget and no post-processing. Only the body of vlm_inference(image_path, question, qid) is editable. The validator checks five invariants before any GPU work: markers present, greedy decoding, backbone unchanged, signature intact and valid syntax. Each mutation is one targeted edit by Gemini 2.5 Flash, conditioned on six failures of the parent.

Comment: Where does qid come from, and could it provide unintended information?

Response: qid is the type of the question being asked (Section 3.1), for example "highest y-tick". CharXiv supplies these labels with its own questions, and for the other three benchmarks it is read from the question itself, so it is available whenever the question is. It only tells the program what form of answer is requested. The gold answer is never an input to the deployed program. The mutator sees gold answers only for six training failures during search, and never for test questions.

Comment: What does calling the VLM zero or more times imply for efficiency? Which InternVL3 and SmolVLM2 sizes were used?

Response: "Zero or more calls" is the space the search may explore. The speeds we report are measured end-to-end throughputs of the deployed programs. best_speed runs at 1.24 queries/s, against 0.33 for the seed (3.8$\times$) and 1.20 for the hand-tuned program. The checkpoints are InternVL3-2B (OpenGVLab/InternVL3-2B-hf) and SmolVLM2-2.2B-Instruct (HuggingFaceTB/SmolVLM2-2.2B-Instruct), alongside Qwen3-VL-2B-Instruct. These identifiers are recorded in the saved execution logs and are now stated in the paper.

Thank you again for these concrete suggestions. We are happy to clarify anything further.

Reviewer_PJMnOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents an interesting framework for improving VLM performance on scientific chart analysis through evolutionary optimization. While the results are promising, the evaluation is insufficient to establish the method's effectiveness and practical value compared with stronger existing VLMs and simpler optimization approaches.

Critical strengths

S1. The framework is clearly presented and targets an interesting optimization problem. S2. The reported improvements are promising and suggest potential for further research. S3. The authors provide code and evolution artifacts to support reproducibility.

Critical weaknesses
  1. Limited comparison with strong VLMs. The evaluation focuses on relatively small open-weight models and lacks comparisons with strong VLMs, making it difficult to assess the method's relevance given current chart-understanding capabilities.
  2. Weak baselines: The low initial performance of the seed programs raises concerns about whether the reported gains reflect meaningful advances beyond stronger prompts, manual tuning, or only post-processing.
  3. Insufficient evaluation: The experiments do not adequately isolate the contribution of evolutionary optimization from changes to prompts, token budgets, and answer post-processing.
  4. Potential data leakage: Candidate selection uses holdout performance across generations, while the final results are reported on the full descriptive dataset, including data used during optimization. This raises concerns about selection bias and generalization to unseen data.
Questions for authors
  1. Stronger baselines: Could you evaluate the proposed method against stronger VLMs to establish whether the approach remains beneficial with stronger, state-of-the-art starting models?

  2. Test-set evaluation: Since candidate selection uses holdout performance across generations, could you evaluate the final selected program on a separate, untouched test set that is never used during evolution or model selection?

  3. Component ablation: Could you isolate the contributions of prompt optimization, token-budget adjustment, and post-processing to determine which components account for the reported improvements? This is important to demonstrate how each component affects the system where the baselines are performing very weakly.

  4. Costs: Could you report the total cost of optimization, including model inference, API calls, and computational resources? This is important to show how efficient this method is, as it also improves the baseline.

3 comments

We greatly appreciate the detailed comments and suggestions made by the reviewer, and have accordingly made changes in the revised version, marked in blue.

Comment: The evaluation lacks stronger VLMs. Does the method remain useful with a stronger starting model?

Response: Thank you for asking this, the answer is yes. We ran Qwen3-VL-32B-Instruct on a 1,000-question subset of the official ChartQA test set (500 human-written and 500 automatically generated questions, one per distinct chart), scored with relaxed accuracy. All the programs were selected before test evaluation. Results can be seen as follows:

MethodRelaxed accuracy
Manual wrapper88.0% (880/1,000)
Prompt-only, matched budget89.0% (890/1,000)
First automated edit89.2% (892/1,000)
Evolved program91.5% (915/1,000)

The paired gain is +3.5 points over manual tuning (95% bootstrap CI [+1.7, +5.3] exemplifying 60 questions gained, 25 lost) and +2.5 points over prompt-only optimization (CI [+0.9, +4.1] demonstrating 45 gained, 20 lost). The +2.3 points over the first automated edit is the gain from the continued search.

Comment: The initial programs are very weak. Could stronger prompts or manual tuning explain the improvement?

Response: We thank the reviewer for raising this great point, and we ran this comparison. On the same test subset with Qwen3-VL-2B-Instruct, we added a competent manual wrapper and a budget-matched prompt-only optimizer:

MethodRelaxed accuracy
Manual wrapper59.0% (590/1,000)
Prompt-only, matched budget61.0% (610/1,000)
First automated edit60.2% (602/1,000)
Evolved program66.5% (665/1,000)

The gain is reported to be 7.5 points over manual tuning (75 more correct answers) and 5.5 points over prompt-only optimization (55 more). The first edit only matches prompt-only optimization, so the margin comes from the continued search. Table 1 remains as a separate, seed-relative measurement in strict exact match.

Comment: Selection uses holdout performance, while final evaluation includes optimization data. Could you evaluate on an untouched test set?

Response: Absolutely. We agree that a holdout used for selection is a validation set, and the paper now says that more clearly. Table 4 scored the training plus holdout questions (102 + 26 = 128) and is now labeled accordingly. The 1,000-question test is the untouched evaluation. It was excluded from mutation feedback, candidate selection, prompt tuning and the choice of deployment variant.

Comment: Isolate the contributions of prompts, generation budgets, and post-processing.

Response: We ran this experiments on the same test with Qwen3-VL-2B-Instruct. Each row puts one component of the evolved program back to the manual setting:

VariantAccuracyChange
Full evolved program66.5%-
Manual prompt restored62.3%-4.2
Manual extractor restored63.8%-2.7
Manual token budget restored65.4%-1.1

The prompt contributes the most, then the extractor, then the token budget. These are conditional effects, so we do not add them.

Comment: Report the full cost of optimization, including model inference, API calls, and compute.

Response: We now report search and deployment separately in a new appendix. The search (25 generations, a population of at most 10, Gemini 2.5 Flash as the mutator) took 3 GPU-hours on an A100, including rejected candidates and retries, and is paid once. The deployed programs on the Qwen3-VL-2B/ChartQA case study are:

ProgramDesc. EMRsn. EMQPSOverall
seed38.3%4.7%0.330.061
hand-tuned56.3%17.2%1.200.198
evolved58.6%9.4%0.780.138
best_accuracy58.6%10.9%0.850.148
best_speed58.6%14.1%1.240.205
best_overall57.0%9.4%1.220.201

best_speed is 3.8$\times$ faster than the seed and about 3.3% faster than the hand-tuned program, so the reason to run the search is the accuracy of the reused program, together with the manual tuning it replaces. On reasoning exact match, the hand-tuned program leads, and we report that as a loss.

We are really grateful for the feedback, which has made the paper more precise. We hope these changes address your concerns, and we are determined to clarify anything further.

Thank you to the authors for the detailed response and the substantial effort put into these new experiments. I appreciate the clear progress made in addressing my earlier concerns—specifically adding the manual and prompt-only baselines, evaluating on a strictly untouched 1,000-question test split, isolating component contributions through ablations, and reporting compute costs.

My main remaining reservation concerns the practical motivation of state-of-the-art models. While Qwen3-VL-32B is a solid addition over 2B, it is still a mid-sized open-weight model rather than a flagship frontier VLM. To establish that this task still leaves meaningful headroom for wrapper-level optimization, the authors do not need to run the full evolutionary search on a frontier model; simply evaluating standard manual prompting of at least one flagship VLM (e.g., GPT 5>, Gemini 3>, Claude 5>) under this exact evaluation setup would clarify whether strong baseline VLMs already saturate the benchmark or genuinely struggle here.

If you can add that baseline comparison showing that frontier VLMs still leave clear room for improvement on this benchmark, I will be happy to raise my score.

We sincerely thank the reviewer for the encouraging feedback and prompt response. We ran the requested additional experiment and added it to the revised version of the paper, marked in blue.

We evaluated two frontier models, 1) GPT-5 and 2) Gemini 3.1 Pro with fixed manual prompting on the same 1,000-question ChartQA test subset, with the same images, questions, answer normalization and relaxed-accuracy evaluator. Each model generated one scored response per question, with no search or prompt selection on the test set. The following table has the results:

ModelProgramRelaxed accuracy
GPT-5Manual prompting89.8% (898/1,000)
Gemini 3.1 ProManual prompting93.8% (938/1,000)
Qwen3-VL-32B-InstructManual wrapper88.0% (880/1,000)
Qwen3-VL-32B-InstructEvolved program91.5% (915/1,000)

We made two observations with these results: 1) frontier models do not saturate this benchmark. As noted, GPT-5 misses 102 questions and Gemini 3.1 Pro misses 62. 2) The evolved program takes the open-weight 32B model from below GPT-5 to above it (91.5% against 89.8%), and fills more than half of the gap to Gemini 3.1 Pro (from 5.8 points to 2.3), with no change to the weights.

We do not claim that our method improves these frontier models, since we evaluated only their manual-prompting baselines due to lack of time. Our evidence for gains from program optimization remains the matched comparisons on Qwen3-VL-2B and Qwen3-VL-32B.

We are truly grateful for this suggestion. This has made the motivation of our paper more precise. We hope it addresses your remaining concern, and we are happy to clarify anything further. Thank you so much for the prompt response and the opportunity again.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.