acceptodds
Withdrawn from ICLR 2027

GeoEvolve: Recursive Skill Evolution for Image Geolocalization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Image geolocalization with large vision-language models (LVLMs) can benefit from external evidence gathered through tools when visual reasoning alone is insufficient to resolve geographic ambiguity. However, unconstrained tool use often introduces distracting or conflicting signals. We introduce , a framework that recursively evolves , a single geolocalization skill encoding an explicit orchestration policy. induces an initial policy from cold-start trajectories collected under minimally constrained tool use, and then enters an on-policy loop that generates new trajectories under its current guidance. At each iteration, assigns process-level credit to effective and harmful decisions within a trajectory batch, and uses this feedback to preserve, clarify, or revise rules for hypothesis formation, tool selection, evidence verification, and stopping. On *Geo-EVO-Test* and *Im2GPS3K*, consistently improves over unconstrained tool use and remains effective across LVLMs of different scales and families. Distilling skill-guided trajectories into a task-specific model yields further gains and sets a new state-of-the-art among compact models in image geolocalization. The code, data, and skills will be publicly released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image Geolocalization, Agentic Reasoning, Recursive Skill Evolution

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_UH2bOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes GeoEvolve, a framework for image geolocalization that iteratively optimizes an external natural-language skill, GeoSkill, to improve hypothesis formation, tool use, evidence verification, and stopping decisions. The authors first use a frozen Qwen3.5-9B model equipped with web search and image retrieval to generate geolocalization trajectories. GLM-5.3-Flash then analyzes effective and harmful decisions within these trajectories to construct and progressively revise the skill, retaining useful updates through evaluation on a development set. This stage does not modify the execution model’s parameters. The authors further fine-tune Qwen3.5-9B on 5,000 skill-guided trajectories to obtain GeoEvolve-9B. Experiments on GEO-EVO-TEST and IM2GPS3K evaluate the effects of skill guidance, recursive evolution, and trajectory-based fine-tuning. On GEO-EVO-TEST, accuracy within 1 km increases from 14.61% with tools alone to 20.40% with GeoSkill, and further to 22.87%*after fine-tuning. The central contribution is to transform experience from individual geolocalization trajectories into an explicit policy that can be continually refined, reused, and used to support model training.

Critical strengths
  • Tool-use experience is stored in an inspectable textual policy, allowing updates without repeatedly fine-tuning the execution model.
  • The framework attempts to distinguish beneficial actions, non-compliance with existing rules, deficiencies in the rules themselves, and external tool failures, providing more fine-grained feedback than judging entire trajectories solely by their final outcomes.
Critical weaknesses
  1. Insufficient discussion of and comparison with closely related work.
    The paper does not cite Skill-Conditioned Visual Geolocation for Vision-Language Models, which already explores external natural-language skills, autonomous skill evolution driven by successful and failed trajectories, and policy improvement without updating model parameters for visual geolocation. More broadly, the coverage and citation of relevant geolocation research are insufficient.

  2. Contradictory descriptions of the experimental protocol.
    The main text repeatedly emphasizes that GeoSkill is evolved once using trajectories from Qwen3.5-9B and then applied to other models without modification. However, the footnote of Table 3 on page 8 explicitly states: “Each GeoSkill is evolved separately for its corresponding base model.” Directly transferring a shared skill and evolving a separate skill for each model are fundamentally different experimental settings. This inconsistency directly affects the interpretation and validity of the paper’s conclusions.

  3. Insufficient experimental validation of a central claim.
    The authors emphasize process-level attribution, but the existing ablations mainly examine tool availability, cold-start initialization, and subsequent evolution. They do not adequately compare the following alternatives under matched experimental conditions:

    • Updating skills based solely on final outcomes.
    • Updating skills through standard trajectory reflection.
    • Updating skills using the proposed process-level feedback.
Questions for authors
  1. Which skill evolution protocol was actually used in the cross-model experiments?
    The main text states that the same GeoSkill is transferred to other models without modification, whereas the footnote of Table 3 states that a separate skill is evolved for each model. Which setting was actually used? Could the authors report results for both direct transfer and separate evolution for each target model?

  2. Does process-level feedback outperform standard trajectory reflection?
    Under the same initial skill, evolution data, and computational budget, could the authors compare updates based solely on final outcomes, standard trajectory reflection, and the proposed process-level feedback? How do the authors validate that the feedback correctly distinguishes failures to follow the skill instructions, flaws in the skill rules, and external tool failures?

  3. What denominator and aggregation procedure are used to compute accuracy?
    GEO-EVO-TEST contains 1,000 samples. If each sample is evaluated once, accuracy should change in increments of 0.1 percentage points. How were values such as 6.56% and 14.61% calculated? In addition, Figure 3 identifies the results with zero on-policy trajectories as those of the cold-start skill, yet some threshold accuracies appear substantially different from the cold-start row in Table 2. For example, Table 2 reports an accuracy of 33.91% at 25 km, whereas the corresponding starting point in Figure 3 appears to be approximately 39%–40%. Could the authors clarify this discrepancy?

  4. Could the authors provide the complete information needed to reproduce skill evolution?
    Could the authors release the final GeoSkill, the changes between key versions, the evolution prompts, the development set size, the update batch size, and the acceptance and rollback criteria? In particular, how is an update accepted or rejected when performance improves at some distance thresholds but declines at others?

I would be willing to raise my score if the authors satisfactorily address these concerns through clear explanations and supporting evidence.

2 comments

We thank the reviewer for the careful evaluation and constructive feedback, particularly the recognition of our inspectable textual skill and process-level feedback. We address the concerns below.

Comparison with Closely Related Geolocalization Work (W1)

We thank the reviewer for highlighting [1]. Both works evolve natural-language skills for visual geolocalization, but differ in their skill representations and evolution mechanisms. The referenced work maintains a Skill-Graph of atomic skills that are retrieved and composed for geographic reasoning, whereas GeoEvolve recursively refines a unified GeoSkill for orchestrating hypothesis formation, tool selection, evidence verification, and stopping.

While [1] synthesizes, merges, and prunes atomic skills using successful and failed trajectories, GeoEvolve uses process-level credit assignment to distinguish agent non-compliance, policy deficiencies, and external tool failures, enabling targeted policy revisions. We have added [1] and clarified these distinctions in Related Work.

[1] Skill-Conditioned Visual Geolocation for Vision-Language Models, arXiv 2026

Clarification of the Cross-Model Transfer Protocol (W2 & Q1)

We apologize for the inconsistent Table 3 footnote, which has been corrected. GeoSkill was evolved solely from Qwen3.5-9B trajectories and transferred unchanged to Qwen3.5-27B and GLM-5.3-Flash, without model-specific evolution. The correction does not affect the results. Our experiments specifically assess cross-model reuse of a single skill. Separately evolving a skill for each backbone is a different setting beyond the scope of this evaluation.

Effectiveness of Process-Level Credit Assignment (W3 & Q2)

We compare GeoEvolve with Outcome-only Evolution and Reflexion [1], representing outcome-based skill refinement and trajectory reflection, respectively, alongside ExpeL [2], TextGrad [3], and GEPA [4]. All methods use Qwen3.5-9B, identical tools, and comparable computational budgets on Geo-EVO-Test.

Method1 km25 km200 km750 km2500 km
Reflexion (NeurIPS'23) [1]19.9738.8042.1049.0068.50
ExpeL (AAAI'24) [2]19.3038.7041.0048.2067.40
TextGrad (Nature'25) [3]15.6337.1041.3048.5066.90
GEPA (ICLR'26) [4]16.8039.1043.2049.5069.00
Outcome-only Evolution19.5535.8347.3757.5974.03
GeoEvolve (Ours)20.4036.3247.6257.9574.53

Effectiveness of Process-Level Feedback. GeoEvolve consistently outperforms Outcome-only Evolution, albeit by modest margins, and achieves the highest accuracy at four of five thresholds among the compared methods, with larger advantages at coarser scales. These results support the value of process-level feedback beyond outcome-based refinement and general-purpose self-improvement.

[1] Reflexion: Language Agents with Verbal Reinforcement Learning, NeurIPS 2023
[2] ExpeL: LLM Agents Are Experiential Learners, AAAI 2024
[3] Optimizing Generative AI by Backpropagating Language Model Feedback, Nature 2025
[4] GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning, ICLR 2026

Qualitative Analysis of Failure Attribution and Skill Revision. We further trace actual GeoSkill revisions across accepted checkpoints to illustrate how process-level feedback leads to targeted policy updates.

(1) Geographic Evidence Verification.

- Use image_retrieval to compare the returned similarity ordering, countries, and coordinate distances; favor a country cluster over isolated neighbours.
+ When retrieval hits span multiple countries and geographically distant locations, treat them as mixed evidence.
+ Do not commit to a city based on an isolated top-1 match; require corroboration from multiple retrieval hits or readable in-image text.

(2) External Tool Failures.

- Use web_search for readable text clues or competing countries; judge literal result snippets.
+ Build queries from image clues rather than location hypotheses.
+ Treat empty, errored, or uninformative results as no geographic evidence.

(3) Refining the Stopping Policy.

- When retrieval hits span multiple countries, require corroboration before committing to a city.
+ Check whether retrieval hits are dispersed across distinct places, even within the same country.
+ Before finalizing, verify that the predicted locality is supported by multiple retrieval hits or readable in-image text.
+ Otherwise, avoid unsupported city-level commitments.

These examples illustrate how GeoEvolve distinguishes agent non-compliance, policy deficiencies, and external tool failures to guide targeted revisions of geographic investigation rules rather than indiscriminately updating GeoSkill. Together with the quantitative comparisons, they provide further evidence for the benefits of process-level credit assignment.

Clarification of Evaluation Metrics and Experimental Results (Q3)

Accuracy Calculation. Although Geo-EVO-Test contains 1,000 images, localization accuracy is calculated only over successfully completed runs. A run is considered completed if a valid geographic coordinate (latitude and longitude) can be extracted from its final output without encountering model request errors or tool execution failures. Specifically, runs are classified as incomplete in the following cases:

  • Model request errors: Timeouts or API errors caused by requesting the underlying model.
  • Tool execution failures: Errors or exceptions occurring when calling tools.
  • Failure to extract valid coordinates: The final output lacks parseable latitude and longitude format.

Accuracy is computed over completed runs using the Haversine distance between predicted and reference coordinates at thresholds of 1, 25, 200, 750, and 2,500 km. Since incomplete runs are excluded, the evaluation denominator may be smaller than the 1,000 images in Geo-EVO-Test, explaining values such as 6.56% and 14.61%. We have clarified the completion criteria and accuracy calculation procedure in the revised manuscript.

Figure 3 vs. Table 2. As stated in Section 4.1, “a separate development split is used for checkpoint selection”. Figure 3 reports the performance of successive GeoSkill checkpoints on this development split, whereas Table 2 reports results on Geo-EVO-Test. Thus, the numerical differences arise from different evaluation splits rather than inconsistent experimental results. We have updated the caption of Figure 3 in the revised manuscript to explicitly clarify this distinction.

Reproducibility and Skill Update Criteria (Q4)

We thank the reviewer for emphasizing reproducibility. The complete implementation details, including the skill evolution and update procedures, are available in our anonymous repository (https://anonymous.4open.science/r/geoevolve-5E5F/).

Skill Revision and Validation. During on-policy evolution, each trajectory batch produces a process-level feedback report, which guides a conservative revision of the current GeoSkill while preserving unaffected policy rules. Each candidate is then evaluated on a fixed, held-out development split against both the initial cold-start skill and the currently accepted checkpoint.

Acceptance and Rollback Criteria. For checkpoint selection, accuracy is calculated over all development images, counting missing or invalid predictions as failures to ensure consistent comparisons. We evaluate each candidate using a weighted combination of localization accuracies at 1, 25, 200, 750, and 2500 km. A candidate is accepted only if it improves the overall performance over the currently accepted checkpoint while satisfying regression safeguards relative to the initial skill. These safeguards prevent substantial degradation at individual geographic scales, protect previously successful cases, and limit severe regressions and invalid predictions. Otherwise, the candidate is rejected, and the previous checkpoint is retained.


We thank the reviewer again for the valuable feedback. We have incorporated the additional experimental results and corresponding discussions into the revised manuscript. The code, data, and evolved skills are also publicly available at https://anonymous.4open.science/r/geoevolve-5E5F/ to facilitate reproducibility. We welcome any remaining concerns and would be happy to discuss them further. If our responses have satisfactorily addressed your concerns, we would greatly appreciate your reconsideration of the rating.

Reviewer_dUBAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GeoEvolve, a framework that introduces recursive self-improvement into tool-augmented image geolocalization. The central idea is to iteratively evolve a reusable GeoSkill while keeping the underlying large vision-language model (LVLM) fixed.

Specifically, GeoEvolve initializes GeoSkill from cold-start trajectories and progressively refines it using process-level credit assignment derived from on-policy experience. The resulting skill can be transferred across different LVLM backbones. Furthermore, the authors explore distilling skill-guided behaviors into training trajectories to further improve the performance of task-specific models.

Experiments on MP16-Evo-Test and Im2GPS3K demonstrate improvements across different LVLM backbones and geographic scales. Overall, the paper explores an interesting direction and provides encouraging empirical results. However, I have concerns regarding the limited comparisons with existing self-improving and agent harness frameworks, as well as the lack of clearly demonstrated geolocalization-specific methodological contributions.

Critical strengths
  1. An interesting application of recursive self-improvement. The paper introduces the concept of recursive self-improvement into image geolocalization, exploring how a model's task-solving capabilities can be progressively enhanced through the evolution of external skills rather than updating the underlying model parameters. I find this direction interesting and potentially valuable.

  2. Competitive performance against representative baselines. The authors compare GeoEvolve against multiple baselines, including a tool-augmented Qwen3.5-9B configuration. The reported improvements provide encouraging evidence that the proposed framework is effective, particularly when compared with a strong tool-using LVLM baseline.

  3. Extensive experimental validation. The paper conducts a reasonably comprehensive set of experiments to evaluate the proposed framework, including evaluations across different LVLM backbones and geographic scales. The additional experiments on skill-guided model training further support the potential utility of the evolved skills.

Critical weaknesses
  1. Insufficient comparison with existing self-improving frameworks. Fundamentally, GeoEvolve appears to be a recursive self-improving framework applied to image geolocalization. However, the experimental comparisons seem to focus primarily on conventional geolocalization methods and tool-augmented LVLM baselines, rather than other self-improving approaches. I believe comparisons with representative self-improving or automatic skill-evolution frameworks would be particularly important for establishing the methodological advantages of GeoEvolve. Without such comparisons, it is difficult to determine whether the reported improvements arise from the specific design of GeoEvolve or from the general benefits of iterative skill refinement. Since recursive self-improvement is not my primary research area, I would also be interested in considering the perspectives of other reviewers with more expertise in this direction. I may update my assessment after reading their reviews and the authors' rebuttal.

  2. Lack of comparison with alternative agent harness frameworks. Since GeoEvolve introduces and iteratively refines an external GeoSkill to guide an LVLM's tool-using behavior, the resulting system can also be viewed as an agent operating within a skill-augmented harness. From this perspective, I believe the paper would benefit from comparisons with alternative agent harness designs. For example, how does GeoEvolve compare with a carefully designed static skill, a manually optimized agent workflow, or other frameworks that support persistent instructions, tool-use strategies, and iterative experience-based refinement? Such comparisons would help distinguish the contributions of the proposed recursive evolution mechanism from the general advantages of providing an LVLM with a well-designed agent harness.

  3. Unclear geolocalization-specific methodological contributions. Although the paper is framed around image geolocalization, I find it difficult to identify which components of the proposed framework are specifically designed for this task. The overall pipeline—initializing a skill from successful trajectories, refining it through process-level feedback, and transferring the resulting behavior across models—appears largely task-agnostic. In principle, a similar framework could potentially be applied to mathematical reasoning, coding, or other tool-augmented problem-solving tasks with relatively minor modifications. While general applicability is not inherently a weakness, the paper should more clearly articulate what makes GeoEvolve particularly suitable for geolocalization and whether it introduces any domain-specific insights beyond applying a general self-improvement paradigm to a new task.

Questions for authors
  1. On comparisons with self-improving frameworks: Could the authors explain how GeoEvolve differs from existing recursive self-improvement and automatic skill-evolution frameworks? In particular, have the authors considered adapting representative methods to the same geolocalization setting under comparable computational budgets and tool-access conditions?

  2. On the role of the agent harness: How much of the performance improvement comes from recursive GeoSkill evolution, as opposed to simply providing the LVLM with a well-designed external skill or agent harness? Could the authors compare GeoEvolve with static skills, manually optimized instructions, or alternative harness designs under the same backbone and tool configurations?

  3. On geolocalization-specific design: Which components of GeoEvolve are specifically tailored to image geolocalization? Would the same framework be directly applicable to other domains, such as mathematical reasoning or coding, without substantial modifications? If so, what constitutes the primary methodological contribution beyond general-purpose recursive skill evolution?

  4. On the effectiveness of recursive evolution: Could the authors provide a more detailed analysis of the performance gains across successive evolution iterations? In particular, how does performance scale with the number of refinement rounds, and how much of the improvement can be attributed to additional inference-time computation or accumulated experience rather than the proposed process-level credit assignment mechanism?

2 comments

We sincerely thank the reviewer for the careful evaluation and constructive feedback. We also appreciate the recognition of our research motivation, the effectiveness of the proposed framework, and the comprehensive experimental validation. We address the concerns below.

Comparison with Existing Self-Improving Frameworks (W1 & Q1)

We thank the reviewer for this valuable suggestion. To the best of our knowledge, GeoEvolve is among the first efforts to integrate tool-augmented LVLMs with recursive skill evolution for image geolocalization. Our primary motivation is to address the lack of effective tool orchestration in relatively lightweight LVLMs, where access to external tools alone does not guarantee effective geographic investigations. Accordingly, our original evaluation primarily focused on comparisons with existing geolocalization approaches. Nevertheless, we agree that comparisons with established self-improving frameworks are important for a more comprehensive assessment of our method, and have therefore conducted additional experiments with four representative approaches.

  1. Methodological Differences. GeoEvolve recursively evolves GeoSkill, a unified and reusable orchestration policy tailored to the geographic ambiguity and heterogeneous evidence encountered in image geolocalization. Rather than relying solely on trajectory-level reflection or generic textual optimization, it uses process-level credit assignment to identify unreliable geographic evidence, premature location commitments, and ineffective tool decisions, while distinguishing agent non-compliance from policy deficiencies to guide targeted revisions of GeoSkill rules.

  2. Comparison with Existing Frameworks. We adapt Reflexion [1], ExpeL [2], TextGrad [3], and GEPA [4] to the image geolocalization setting. All methods are evaluated on Geo-EVO-Test with the same Qwen3.5-9B backbone, external tools, and comparable computational budgets. | Method | 1 km | 25 km | 200 km | 750 km | 2500 km | |:--|--:|--:|--:|--:|--:| | Reflexion (NeurIPS'23) [1] | 19.97 | 38.80 | 42.10 | 49.00 | 68.50 | | ExpeL (AAAI'24) [2] | 19.30 | 38.70 | 41.00 | 48.20 | 67.40 | | TextGrad (Nature'25) [3] | 15.63 | 37.10 | 41.30 | 48.50 | 66.90 | | GEPA (ICLR'26) [4] | 16.80 | 39.10 | 43.20 | 49.50 | 69.00 | | Outcome-only Evolution | 19.55 | 35.83 | 47.37 | 57.59 | 74.03 | | GeoEvolve (Ours) | 20.40 | 36.32 | 47.62 | 57.95 | 74.53 |

The results demonstrate that GeoEvolve achieves stronger overall geolocalization performance than these representative self-improving frameworks, with particularly notable improvements at broader geographic scales. This suggests that process-aware, targeted skill evolution is beneficial for improving tool-augmented geographic investigations.

[1] Reflexion: Language Agents with Verbal Reinforcement Learning, NeurIPS 2023
[2] ExpeL: LLM Agents Are Experiential Learners, AAAI 2024
[3] Optimizing Generative AI by Backpropagating Language Model Feedback, Nature 2025
[4] GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning, ICLR 2026

Benefits of Recursive Evolution Beyond Static Agent Harnesses (W2 & Q2)

We agree that providing an LVLM with a well-designed external skill or workflow can itself improve tool-use behavior. To distinguish this effect from the benefits of recursive skill evolution, we conduct additional comparisons with Human-Written Prompt, and Fixed Workflow, alongside the Tool-only baseline. All methods use the same Qwen3.5-9B backbone and external tools and are evaluated on GEO-EVO-TEST.

Method1 km25 km200 km750 km2500 km
Tool-only14.6131.0143.7051.4568.80
+ Human-Written Prompt16.3234.9445.7756.8776.42
+ Fixed Workflow16.6234.0545.1757.1074.42
+ Skill (Outcome-only Evolution)19.5535.8347.3757.5974.03
+ GeoSkill (Ours)20.4036.3247.6257.9574.53

The results show that static prompts and predefined workflows offer improvements over direct tool use, confirming the value of external guidance. However, GeoSkill achieves stronger overall performance, particularly at broader geographic scales, suggesting that its benefits extend beyond those of a static agent harness. The comparison with Outcome-only Evolution further supports the value of process-level feedback in refining the orchestration policy.

Geolocalization-Specific Design and Contributions (W3 & Q3)

We thank the reviewer for this important question. While recursive skill evolution is potentially generalizable, GeoEvolve's process-level feedback and skill evolution are tailored to the challenges of tool-augmented image geolocalization.

Geographic Investigation. GeoSkill guides geographic hypothesis formation from visual clues, selection between Web Search and Image Retrieval, geographic consistency verification, and investigation stopping. These decisions are particularly important for lightweight LVLMs that struggle to coordinate heterogeneous geographic evidence.

Domain-Aware Process Credit Assignment. Unlike outcome-level feedback based solely on geographic prediction errors, our approach examines why an investigation succeeds or fails. Specifically, it analyzes retrieval similarity and geographic dispersion, confirmation-biased searches, premature geographic commitments, and unnecessary tool calls. Crucially, it distinguishes failures caused by non-compliance with existing GeoSkill rules from those caused by deficiencies in the rules themselves, allowing feedback to be assigned to the relevant decisions rather than treating an entire trajectory as a single success or failure.

Task-Specific Policy Revision. These diagnoses are translated into actionable geographic investigation rules. For example, unreliable retrieval decisions can motivate replacing rigid similarity thresholds with evidence-aware criteria based on relative similarity and geographic clustering, while confirmation-biased searches motivate explicit cross-verification and hypothesis elimination. Such targeted revisions improve how GeoSkill coordinates tools and evaluates geographic evidence, rather than merely accumulating additional instructions.

Thus, our contribution connects domain-aware process credit assignment with the evolution of an explicit geographic investigation policy. While the framework may generalize to other domains, its effectiveness in image geolocalization relies on task-specific evidence analysis and policy design.

Evolution Dynamics and Performance Attribution (Q4)

To analyze recursive skill evolution, we evaluate successive checkpoints on the fixed development split used in Figure 3 and report their inference-time statistics.

Checkpoint1 km25 km200 km750 km2500 kmTool CallsTotal StepsLatency (s)Gate Decision
00 (Cold-Start)18.7139.4649.6662.2176.171.257.8817.3Baseline
0318.7140.1650.6762.3975.901.298.1414.9Rejected
0619.0740.1649.1662.5676.981.508.7013.8Rejected
0919.4340.1650.1662.6576.711.689.3712.2Accepted
1220.1542.2751.6762.7477.521.558.8913.5Accepted
1520.5140.8651.1762.5675.901.548.8613.5Rejected
1820.1539.4649.1662.3076.441.428.3715.0Rejected
1919.0738.0546.6461.5972.661.368.2515.4Rejected

Tool Calls, Total Steps, and Latency are averaged per image.

Evolution Dynamics. Checkpoint 12 improves over Cold-Start across all five geographic thresholds by 0.53–2.81 percentage points. However, performance is not monotonic: Checkpoint 15 achieves higher 1 km accuracy (20.51% vs. 20.15%) but performs worse at other scales, highlighting the importance of development-set validation.

Inference-Time Computation. Compared with the previously accepted Checkpoint 09, Checkpoint 12 achieves higher accuracy at all five thresholds with fewer tool calls (1.55 vs. 1.68) and total steps (8.89 vs. 9.37). Thus, more inference-time interactions do not necessarily yield better localization.

Accumulated Experience and Process-Level Feedback. As shown in the preceding comparisons, GeoEvolve achieves stronger overall performance than Outcome-only Evolution under comparable evolution data and computational budgets. These results suggest that the gains extend beyond accumulating more trajectories, highlighting the value of process-level credit assignment in guiding targeted skill revisions.

We have clarified the development-set evaluation in Figure 3 and added the checkpoint analysis to the revised manuscript.


We thank the reviewer again for the valuable feedback. We have incorporated the additional experimental results and corresponding discussions into the revised manuscript. The code, data, and evolved skills are also publicly available at https://anonymous.4open.science/r/geoevolve-5E5F/ to facilitate reproducibility. We welcome any remaining concerns and would be happy to discuss them further. If our responses have satisfactorily addressed your concerns, we would greatly appreciate your reconsideration of the rating.

Reviewer_EgFROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

GeoEvolve uses web search and geotagged-image retrieval to locate photos. A text policy, GeoSkill, tells a vision-language model how to use both tools; another model updates that policy from batches of attempts. The authors report gains on Geo-Evo-Test and Im2GPS3K and transfer to other models.

Critical strengths

The cold-start policy is useful. On Qwen3.5-9B, adding unguided web search hurts image retrieval, but the initial skill raises 1 km accuracy from 14.61% to 18.94% (Table 2). On Im2GPS3K, the skill-guided system also beats prior tool-based systems at street level by about six points (Table 1).

Critical weaknesses

Figure 3 and Table 2 disagree despite supposedly showing the same Geo-Evo-Test run. At 25 km, the cold-start point looks near 39.5 in the figure but is 33.91 in the table; the peak is also about six points higher. Similar gaps appear at coarser distances. The plotted split and checkpoint selection are unclear.

The refinement gain may just be fewer unfinished runs. Completion rises from 90% to 96% (Table 7). Dividing Table 2 accuracy by completion rate gives about 21% for both skills at 1 km, with the refined skill slightly worse at coarser thresholds. Those extra completed runs explain the reported gains without better geolocation. The paper never defines completion.

Questions for authors
  • Which split does Figure 3 plot, and which checkpoint do Tables 1 to 4 use?
  • What is a completed run in Table 7, and what are the Table 2 accuracies on images all three skill variants completed?
  • The Table 3 footnote says each skill was evolved per base model; Section 4.1 says one skill was reused unchanged. Which is true?
  • Values such as 6.56 cannot arise from 1,000 images. Are the numbers averaged over runs?
  • We might need a new experiment. Table 4 changes both training data and test-time prompt. Please add the model fine-tuned on tool-only runs with the skill at inference, and the skill-trained GeoEvolve-9B without it.
2 comments

We sincerely thank the reviewer for the careful evaluation and constructive feedback. We also appreciate the recognition of the effectiveness of our cold-start skill and the encouraging experimental results. We address the concerns below.

Clarification of Evaluation Splits and Checkpoint Selection (W1 & Q1)

We thank the reviewer for pointing out the discrepancy between Figure 3 and Table 2 and apologize for the confusion caused by the unclear presentation. Figure 3 reports the performance of successive skill checkpoints on this development split, whereas Table 2 reports results on Geo-EVO-Test. The GeoSkill results in Tables 1–4 use the same final checkpoint selected through development-set evaluation. Thus, the numerical differences arise from different evaluation splits rather than inconsistent results. We have updated the caption of Figure 3 in the revised manuscript to explicitly clarify this distinction.

Impact of Run Completion on Localization Accuracy (W2, Q2 & Q4)

  1. Definition of Completion and Accuracy Calculation. A run is considered completed if it terminates without model request or tool execution failures and produces a valid geographic coordinate. Specifically, a run is classified as incomplete if it encounters any of the following failure modes:
    • Model request errors: Timeouts or API errors caused by requesting the underlying model.
    • Tool execution failures: Errors or exceptions occurring when calling tools.
    • Failure to extract valid coordinates: The final output lacks parseable latitude and longitude format.

Our reported localization accuracy is calculated exclusively over these successfully completed runs using the Haversine distance between predicted and reference coordinates at thresholds of 1, 25, 200, 750, and 2,500 km. Since incomplete runs are excluded, the number of evaluated samples may differ from the 1,000 images in Geo-EVO-Test, explaining values such as 6.56%. We have clarified the completion criteria and accuracy calculation procedure in the revised manuscript.

  1. Impact of Completion on Performance Gains. To address the concern that the improvements may simply result from fewer incomplete runs, we compare three skill variants (Cold-Start, On-Policy Only, and GeoSkill) on Geo-EVO-Test. Specifically, we randomly sample 500 images from the common subset of successfully completed runs across all three variants, ensuring that localization accuracy is evaluated on exactly the same samples. | Method | 1 km | 25 km | 200 km | 750 km | 2500 km | |:--|--:|--:|--:|--:|--:| | Cold-Start | 18.4 | 39.6 | 50.2 | 60.8 | 76.2 | | On-Policy Only | 19.2 | 40.4 | 51.0 | 62.6 | 78.0 | | GeoSkill (Ours) | 20.2 | 40.8 | 51.0 | 63.2 | 78.2 |

The results show that GeoSkill improves localization accuracy over Cold-Start across all geographic scales and matches or outperforms On-Policy Only on the same evaluation samples. Furthermore, GeoSkill increases the completion rate from 90% to 96%, indicating improvements in both localization accuracy and task completion reliability.

Clarification of the Cross-Model Transfer Protocol (Q3)

We thank the reviewer for pointing out this inconsistency and apologize for the confusion. We clarify that GeoSkill was iteratively evolved using Qwen3.5-9B trajectories and then transferred unchanged to Qwen3.5-27B and GLM-5.3-Flash, without model-specific evolution. The footnote in Table 3 was inadvertently left unchanged during manuscript revisions and has now been corrected in the revised manuscript. This does not affect the reported results.

Disentangling Training-time and Inference-time Contributions (Q5)

To disentangle the contributions of skill-guided training and inference-time guidance, we extend Table 4 with a controlled 2×2 comparison on Geo-EVO-Test. Specifically, we fine-tune Qwen3.5-9B using either tool-only or GeoSkill-guided trajectories and evaluate both models with and without GeoSkill at inference time.

Training TrajectoriesInference Policy1 km25 km200 km750 km2500 km
Tool-onlyTool-only19.7839.2048.8960.1076.15
Tool-onlyGeoSkill17.3435.0645.1356.3073.22
Skill-guidedTool-only16.0233.6843.3554.5871.43
Skill-guidedGeoSkill22.8741.9251.7563.6378.05

Findings and Implications. Both mismatched configurations lead to performance degradation across all geographic scales, whereas skill-guided SFT with consistent GeoSkill guidance achieves the best results. This may reflect training–inference mismatch, as task-specific SFT adapts models to the prompting and tool-use strategies encountered during training. In contrast, our main experiments apply GeoSkill to general-purpose LVLMs without task-specific fine-tuning, where it provides effective inference-time guidance.


We thank the reviewer again for the valuable feedback. We have incorporated the additional experimental results and corresponding discussions into the revised manuscript. The code, data, and evolved skills are also publicly available at https://anonymous.4open.science/r/geoevolve-5E5F/ to facilitate reproducibility. We welcome any remaining concerns and would be happy to discuss them further. If our responses have satisfactorily addressed your concerns, we would greatly appreciate your reconsideration of the rating.

Reviewer_PJciOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

To prevent conflicting or distracting external tool signals from degrading worldwide image geolocalization, the authors introduce GeoEvolve. GeoEvolve treats a natural-language skill file (GeoSkill) as a textual policy checkpoint that guides hypothesis formation, tool selection, evidence verification, and stopping. Starting from cold-start trajectories generated by a frozen source model, an evolution model assigns five-level process credit to recurring decision patterns. In subsequent on-policy rounds, negative patterns are diagnosed into four categories to selectively clarify or revise the skill, with candidate updates gated on a validation split. GeoSkill improves over unconstrained tool use, transfers across frozen LVLMs, and enables teacher trajectory distillation to fine-tune a 9B model.

Critical strengths
  1. Formulating skill evolution in textual policy space is clear and well-motivated. The paper shows that naive multi-tool access can hurt accuracy, and addresses this by iteratively refining an explicit SKILL.md checkpoint.
  2. The proposed approach improves both inference-time tool use and trajectory distillation.
Critical weaknesses
  1. The skill evolution essentially hacks the specific evaluation pipeline, and its transferability is unproven. It overfits to the fixed MP-16 retrieval setup, while Table 3's caption contradicts the text by stating skills are evolved separately per model and Table 1 omits tool-only baselines on IM2GPS3K.
  2. Experiments are limited to small or lightweight models, leaving effectiveness on knowledge-rich large LVLMs unproven. The paper only tests Qwen3.5-9B/27B and GLM-5.3-Flash, and on the stronger GLM-5.3-Flash, fine-grained gains already shrink to +0.39% at 1km and +0.12% at 25km.
  3. Core ablations for process-level credit assignment and human-written skills are missing. The paper only compares GeoSkill against using no skill at all, without testing a simple human-written prompt or outcome-only skill revision to justify the proposed credit mechanism.
Questions for authors
  1. Can the evolved skills improve the performance of state-of-the-art models (e.g GPT-6)? If the skills and the knowledge are included in the latest models, the soundness for the paper may be limited.
  2. Why does the caption of Table 3 state that each GeoSkill is evolved separately for its corresponding base model, whereas the main text and Table 1 caption claim that a single GeoSkill evolved solely from Qwen3.5-9B was applied unchanged to Qwen3.5-27B and GLM-5.3-Flash?
2 comments

We sincerely thank the reviewer for the careful evaluation and constructive feedback. We also appreciate the recognition of our motivation and the effectiveness of our proposed framework. We address the concerns below.

Experimental Protocol and Transferability (W1 & Q2)

  1. Clarification of Table 3 Footnote. We thank the reviewer for pointing out this inconsistency and apologize for the confusion. We clarify that GeoSkill was iteratively evolved using Qwen3.5-9B trajectories and then transferred unchanged to Qwen3.5-27B and GLM-5.3-Flash, without model-specific evolution. The footnote was inadvertently left unchanged during manuscript revisions and has now been corrected in the revised manuscript. This does not affect the reported results.
  2. Regarding Pipeline Overfitting. We would like to clarify that using MP-16 as a retrieval database follows an established experimental setting in worldwide image geolocalization. Representative retrieval-based methods, such as G3 [1] and GeoRanker [2], also construct their retrieval databases using MP-16/MP16-Pro and evaluate performance on independent benchmarks, including Im2GPS3K. Therefore, the MP-16 retrieval setup is not specifically designed for GeoEvolve, but follows a commonly adopted protocol in prior research. More importantly, GeoEvolve does not optimize the retrieval database or modify the underlying tools. Instead, it evolves a textual orchestration policy that guides the LVLM in selecting among available tools, evaluating retrieved evidence, and conducting geographic investigations. The evolution and evaluation images are disjoint, and the resulting GeoSkill is evaluated on unseen images without further adaptation. These distinctions are important when assessing whether the framework merely exploits a fixed evaluation pipeline.

[1] G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models, NeurIPS 2024
[2] GeoRanker: Distance-Aware Ranking for Worldwide Image Geolocalization, NeurIPS 2025

  1. Cross-Dataset Evaluation. To further assess the generalizability of GeoSkill beyond the evolution dataset, we conduct additional experiments on Im2GPS3K using Qwen3.5-9B. We compare direct inference, tool-only inference, and GeoSkill-guided inference under the same backbone and tool configurations. | Method | 1 km | 25 km | 200 km | 750 km | 2500 km | |:--|--:|--:|--:|--:|--:| | Qwen3.5-9B | 8.81 | 28.99 | 41.98 | 59.66 | 77.20 | | + Tools | 12.08 | 36.47 | 53.39 | 70.18 | 80.76 | | + GeoSkill | 20.10 | 38.52 | 57.88 | 74.61 | 85.10 | | Δ vs. Tools | +8.02 | +2.05 | +4.49 | +4.43 | +4.34 |

The results show that external tool access substantially improves localization accuracy across multiple geographic scales. More importantly, GeoSkill consistently outperforms tool-only inference at all five distance thresholds, with improvements ranging from 2.05 to 8.02 percentage points. These findings demonstrate that GeoSkill provides additional benefits beyond tool access alone and that the evolved orchestration policy remains effective on Im2GPS3K, an independent benchmark outside the evolution dataset.

Generalizability to Stronger LVLMs (W2 & Q1)

We thank the reviewer for raising this important question. Due to the high API cost of GPT-6, we instead evaluate Kimi-K3.1-Pro (2.8T total parameters, 104B activated), a substantially larger LVLM, on Geo-EVO-Test. We directly apply the GeoSkill evolved from Qwen3.5-9B trajectories without any model-specific adaptation.

Method1 km25 km200 km750 km2500 km
Kimi-K3.1-Pro25.2748.7560.5069.4079.72
+ Tools29.2552.7260.2069.7381.97
+ Tools + GeoSkill25.3348.8356.0068.0083.00

Findings and Implications. While GeoSkill provides substantial gains for Qwen3.5-9B, the improvements become smaller for Qwen3.5-27B and GLM-5.3-Flash, and largely disappear for Kimi-K3.1-Pro, where accuracy decreases at four of the five geographic thresholds. This suggests that the benefits of an evolved orchestration policy depend on the capabilities of the target LVLM. Rules that help lightweight models coordinate geographic investigations may offer limited additional value or even constrain stronger models with more capable reasoning and tool-use strategies.

These findings also highlight a limitation of single-source skill evolution, which may bias the learned policy toward the source model. They motivate future exploration of multi-source skill evolution and model-adaptive orchestration policies to improve cross-model transferability. We have incorporated these results and corresponding discussion into the revised manuscript.

Necessity of Process-Level Credit Assignment (W3)

We appreciate the reviewer's suggestion to compare GeoEvolve with simpler skill construction and evolution strategies. The motivation behind process-level credit assignment is that final localization outcomes alone cannot reliably identify which decisions should inform skill revisions. An unsuccessful investigation may arise from non-compliance with existing instructions, deficiencies in the skill policy, or unreliable external tool evidence. GeoEvolve distinguishes these cases to guide more targeted skill revisions.

To further evaluate the effectiveness of this design, we introduce three additional baselines on GEO-EVO-TEST using Qwen3.5-9B:

  • Human-Written Prompt: A manually designed strategy prompt based on practical geolocation reasoning experience and publicly available geolocation guidelines [3], without iterative refinement.
  • Fixed Workflow: A programmatic implementation of the Human-Written Prompt, where the reasoning and tool-use procedures are explicitly executed through a fixed pipeline.
  • Outcome-only Evolution: Evolves the skill using final localization outcomes without the proposed process-level credit assignment.
Method1 km25 km200 km750 km2500 km
Tool-only14.6131.0143.7051.4568.80
+ Human-Written Prompt16.3234.9445.7756.8776.42
+ Fixed Workflow16.6234.0545.1757.1074.42
+ Skill (Outcome-only Evolution)19.5535.8347.3757.5974.03
+ GeoSkill (Ours)20.4036.3247.6257.9574.53

Both human-written guidance and fixed workflows improve over tool-only inference, confirming the value of explicit orchestration. Outcome-only evolution provides further gains at several geographic scales, demonstrating the benefits of experience-driven skill refinement. Compared with outcome-only evolution, GeoSkill achieves consistent improvements across all five geographic thresholds. Although the additional gains are modest, these results support the value of process-level credit assignment in providing more targeted feedback for skill evolution.

[3] "GeoGuessr – The Top Tips, Tricks and Techniques", https://somerandomstuff1.wordpress.com/2019/02/08/geoguessr-the-top-tips-tricks-and-techniques/


We thank the reviewer again for the valuable feedback. We have incorporated the additional experimental results and corresponding discussions into the revised manuscript. The code, data, and evolved skills are also publicly available at https://anonymous.4open.science/r/geoevolve-5E5F/ to facilitate reproducibility. We welcome any remaining concerns and would be happy to discuss them further. If our responses have satisfactorily addressed your concerns, we would greatly appreciate your reconsideration of the rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revision Summary

Following the reviewers' feedback, we have revised the manuscript to clarify GeoEvolve's methodological contributions, strengthen experimental validation, address concerns about generalizability and reproducibility, and correct two minor inconsistencies in a table note and a figure caption. All revisions are highlighted in blue and summarized below:

  • Clarified distinctions from related skill-evolution work and highlighted geolocalization-specific contributions in Section 2.2.
  • Clarified evaluation protocols in Section 4.1, corrected the note in Table 5, and revised the caption of Figure 3.
  • Added controlled ablations on completion bias, static guidance, and outcome-only evolution in Table 3 and Table 4.
  • Extended training–inference ablations in Table 6 and cross-model transfer evaluation in Table 8.
  • Added comparisons with representative self-improving frameworks in Table 7.
  • Expanded evolution analysis with checkpoint performance and inference-time statistics in Appendix C.2, and added representative skill revisions in Appendix D.
  • Specified checkpoint acceptance criteria in Appendix A.1 and provided an anonymous implementation repository (https://anonymous.4open.science/r/geoevolve-5E5F/).

The revised manuscript and our individual responses provide further details on these changes. We believe the additional experiments and clarifications offer a more comprehensive assessment of GeoEvolve, and we welcome further feedback from the reviewers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.