- Rating
- 4(+2)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 3(+2)
Read review
Paper summary
The paper, though titled as if it approaches geocoding broadly, approaches a narrower task: translating a natural language address of a shop, building, or facility to latitude-longitude coordinates. It creates a large dataset for this task, well balanced across regions of the globe, whose description was initially inadequate but is now sufficient after the author response. It introduces a reasonable approach to the problem, fine-tuning separate LLMs for different regions with different address formatting styles, and while the approach was originally only compared to commercial geocoding APIs and LLMs not trained for the task, it has been extended to comparisons with geocoding models from the published literature during the author response.
Critical strengths
- The proposed framework for generating coordinates from natural language addresses (fine-tune separate chain-of-thought style generators for different regions + fine-tune a region classifier) is sensible, and is justified by the ablations performed in the paper.
- The proposed approach is competitive with commercial geocoding APIs (a couple km worse in median Haversine error on Bench-C and Bench-A, but much better on noisy CRM and OCR data) and frontier language models (similar on Bench-C and Bench-A but twice as far off on Bench-N), while requiring more modest computational resources (only 4B LLMs) than frontier language models.
Critical weaknesses
[These weaknesses have been addressed in the author response.]
- The text (including the appendix) provides insufficient description of how the GeoMosaic dataset is constructed, and there are no measures of dataset quality (e.g., human validation). Since the dataset is claimed as one of the main contributions of the article, and since the dataset is the only evaluation of the proposed models, the main text needs to describe the dataset construction in sufficient detail that it could be replicated, and needs to include some human evaluation that validates the quality of the generated dataset. The same is true for the even more briefly described "CRM address geocoding" and "OCR logistics document geocoding" datasets. See also the many questions about missing dataset construction details below.
- The method is not compared to a sufficient number of baselines. Beyond commercial geocoding APIs (for which the internals are not known), the approach is only compared LLMs not trained for the task, not any "retrieve candidate entities from gazetteers"-style baselines. The authors cite a few older (<=2018) such approaches, and more recent ones exist (e.g., https://aclanthology.org/2024.naacl-short.3/). Some such approach needs to be included as another baseline. This is especially important as the authors make many claims about efficiency, but do not provide an apples-to-apples comparison of their local LLM-direct-generation approach to a local retrieve-from-gazetteer approach.
Questions for authors
[These questions have been addressed in the author response.]
- What is a "structured address"? What are all the components?
- How do you go from an arbitrary OpenStreetMap record to an address, when many OpenStreetMap records do not correspond to addresses (e.g., https://www.openstreetmap.org/relation/51477)?
- What is the distribution of areas for the locations that you're geocoding? I'm trying to understand how reasonable it is to represent each address with a latitude-longitude point. It could be reasonable if you're only trying to geocode buildings. It's not reasonable if you also want to geocode countries, oceans, etc.
- What are the "basic address and geometry quality requirements"?
- How exactly do you define IARs from "writing systems, administrative hierarchies, and regional balance"? From Figure 1, they seem almost random in some places (e.g., the speckling over the USA)
- How are the auxiliary form fields for A5 generated?
- How do you detect if any A5 generation "introduces a new place name, conflicting address component, coordinate, entity identifier, or target hint"?
- How is the N1 dataset generated?
- How is the N2 dataset generated?
- How is the CRM address geocoding dataset generated?
- How is the OCR logistics document geocoding dataset generated?
- What are the definitions of the evaluation measures? (Med.↓ P90↓ @1↑ @5↑ @25↑ etc.)
- How should I see "preserving high regional accuracy while extending global coverage" in Figure 3? Isn't that figure showing that performance drops significantly with the IAR strategy, but not nearly as bad as with the global strategy? E.g., P90 increases by +3.75 with IAR, but by +9.05% with global.
- What is the purpose of the "reasoning process" in the NL-to-address generation component? It's not used anywhere that I can see.
- Why are the large frontier models not evaluated on CRM and OCR tasks?
Not questions, but other things you should consider fixing:
- Please try to put figures and tables closer to the text where they are mentioned. It's difficult to have to page back and forth so much when reading.
- Figures 1 and 2 are never referenced in the text.
- Define "IAR" the first time you use it, on line 194, instead of later, on line 252.
- "Best and second-best results are in bold and underlined" is not true. OpenCage should be bold for Med.↓, @1↑, @5↑, etc. GPT-4.1 should be bold for P90↓, etc.
- Please report base model for GeoExperts (Qwen3.5-4B) in the main text, not just the appendix. Knowing this is critical for reading results tables that compare to the non-fine-tuned Qwen3.5-4B.
10 comments
Response to Reviewer UpYb
W2. Missing comparison with a gazetteer retrieval baseline
Response: We sincerely appreciate this insightful suggestion and your guidance on strengthening the baseline comparisons. We agree that the existing baselines do not sufficiently establish an accuracy advantage over local retrieval-based geocoding and that additional comparisons are needed. GeoPLACE, which you suggested, predicts country, state, and geographic feature type to constrain candidate searches in a geographic knowledge base, making it a relevant and valuable comparison [1]. However, the authors have not released the complete training data needed to directly reproduce GeoPLACE, so we cannot fully reproduce the model. As an alternative, we reproduce the GeoNorm retrieval baseline [3] used by the contemporary GeoPLACE work. Its workflow is as follows:
-
Use the place name in the input as the query, for example, Paris.
-
Retrieve name-matching candidates from a Lucene index of the local GeoNames gazetteer.
-
Generate and rank candidates using exact matching, fuzzy matching, character n-grams, and related methods.
-
Return latitude and longitude.
We evaluate it on Bench-C/A, with the following results:
Table A. GeoNorm retrieval results.
| Benchmark | Median (km) | P90 (km) |
|---|---|---|
| Bench-C (5K) | 336.81 | 1,530.24 |
| Bench-A (25K) | 264.89 | 1,474.69 |
| Overall (30K) | 279.47 | 1,486.52 |
The results show that its performance falls well below that of GeoExperts. We would also like to explain that this type of retrieval baseline cannot directly accept structured addresses, so we design specialized programmatic interfaces for it. For example, consider a query for the place name Springfield with the country specified as the United Kingdom.
We implement the following steps ourselves:
-
Search globally for Springfield.
-
Retrieve at most places with this name.
-
Remove all places outside the United Kingdom from these results.
-
Return the remaining results as the answer.
If is too small, the system may fail to return a valid output.
This also shows the operational complexity and lack of generalizability of retrieval-based geocoding systems, demonstrating the necessity and value of GeoExperts. We also add these experimental results to the main text to facilitate comparison (line 443).
Response to Reviewer UpYb
Q1. What is a structured address, and what components does it contain?
Response: In this paper, a structured address consists of source-traceable fields organized from coarse to fine:
Table B. Structured address fields.
| Field | Required | Content |
|---|---|---|
| Country | Yes | Country or region |
| Administrative areas | No | States, provinces, regions, counties, etc. |
| Locality | Yes | Local units such as cities, towns, and villages |
| Sublocality | No | Neighborhoods, communities, etc. |
| Street | Yes | Street name |
| Named feature | Yes | Named feature to be localized |
For a target entity , its structured address can be formalized as
where denotes the country or region, denotes the administrative areas ordered from outer to inner, denotes the locality, the sublocality, the street, and the named feature to be localized. The administrative-area sequence satisfies
where means that the preceding level geographically contains the following level. Because nearly all countries have naming conventions for cities and streets, we make country, locality, street, and named feature required fields. Administrative areas and sublocality, however, are not administrative structures shared by all countries, so we make them optional. A complete structured address can therefore be expressed as
Here, means that field is nonempty after text cleaning and contains neither invalid control characters nor excessively long content. denotes a missing optional field. Missing optional fields are simply omitted.
In the actual data, addresses follow this fixed hierarchy:
The administrative_areas are ordered geographically from outer to inner. When sublocality is present, it appears between locality and street.
The serialization function can be written as
=c\oplus a_1\oplus\cdots\oplus a_m\oplus\ell\oplus s\oplus r\oplus f,$$ The separator is fixed to an English comma followed by one space, namely $\oplus=\texttt{", "}$. For example, an address can be structured as: ```text United States, California, San Francisco, The Embarcadero, Ferry Building ``` We add this explanation in Appendix B.2 (lines 825–841) to help readers understand the representation. --- ## Q2. How is an address obtained from an arbitrary OSM record? **Response:** We do not convert arbitrary OSM records into addresses. Instead, we select eligible records using the **water-filling sampling** procedure (lines 842–850) and **two-stage data screening** (lines 850–913) in Appendix B.2. In brief, we first require an object to have a valid name and satisfy the target-category rules, such as those for shops and stations, and then verify its address and geometry. **Objects with missing required fields, conflicts, or geometries that fail the constraints are not selected**. The relation/51477 example you mentioned is Germany's national administrative boundary [2]. Its area is too large for it to be a **GeoMosaic** candidate. It is excluded by the first quantitative condition for area targets, $A\leq250{,}000\\ \mathrm{m}^{2}$, in the first stage of data-quality validation in Appendix B.2 (line 883).
Response to Reviewer UpYb
Response to Reviewer UpYb
We sincerely thank Reviewer UpYb for your time, care, and thoughtful feedback. We deeply appreciate your help in improving our work and respond below.
W1. Insufficient details on dataset construction and quality validation
Response: We agree that, as a core contribution, GeoMosaic should be accompanied by construction details sufficient for reproduction. We sincerely apologize for the difficulty in reading caused by the lack of detailed dataset-construction information. To make the process clearer and more reproducible, we add the corresponding construction details in the main text and appendices. We also provide more targeted explanations for each question below.
We expand the description of data-quality validation in Appendix B.2 (lines 850–913). Specifically, the programmatic checks have two stages: screening out ineligible candidates during construction and rechecking after the initial screening. The first checks for missing or invalid information, while the second verifies that the selected examples are consistent with the original data.
1. Data screening
In the first stage of quality validation, we retrieve a target from the original PBF data pool using the numeric ID of its OSM object. PBF stands for Protocolbuffer Binary Format, a binary file format for storing OSM data. The PBF data for each target entity contain attributes such as its country and street. We then apply the following screening steps. First, we check the PBF data for the country, locality, street, and named feature fields; an example is considered eligible only when all four are present. Second, we use programmatic scripts to check its geometry:
-
-
-
A\leq250{,}000\\ \mathrm{m}^{2}, \quad \frac{4\pi A}{P^{2}}\geq0.15, \quad \max_{v\in V_{\mathrm{outer}}}d_{\mathrm{Hav}}(p,v)\leq250\\ \mathrm{m}. \end{gathered}$$ Here, $A$ is the target area, $P$ is its perimeter, $p$ is the computed interior representative point, and $V_{\mathrm{outer}}$ is the set of outer-ring vertices of the component polygons. The first condition excludes targets with excessively large areas; the second excludes targets that are too elongated or dispersed; and the third excludes area targets with excessive spatial extent, requiring the distance from the GPS ground truth to every outer-ring vertex to be no greater than 250 meters. --- ### **2. Data rechecking** The second stage checks for duplication, ambiguity, and leakage. Using programmatic methods such as string matching, we ensure that an example neither appears repeatedly in GeoMosaic nor corresponds to multiple targets. Different OSM objects may also correspond to the same address. For example, an object with node ID 123 and another with way ID 456 may both become “United States, Springfield, Main Street, Central Library” after conversion to structured addresses. This also creates duplication despite the distinct original OSM objects. We therefore perform **address-level deduplication** on the candidates. If one address corresponds to multiple OSM objects, we exclude all associated candidates, preventing the remaining target from being incorrectly regarded as inherently unique when another conflicting target happens to be excluded by other rules. We also conduct manual spot checks. First, we compare the OSM and Wikidata sources to check consistency among address fields, target entities, and factual clues. Second, we compare each standard address with its variants to check whether field deletion, character perturbation, and added information follow the task rules, preserve the target, and introduce any conflicting place. Finally, for N1/N2, assessors first judge whether the query contains sufficient information and whether it is ambiguous without seeing the reference answer. They then verify the target against source evidence and check whether other entities in the specified candidate domain satisfy the same conditions. Two assessors perform the manual checks, **independently record their decisions** and issue categories, and cross-check examples on which they disagree. For the construction of the CRM/OCR datasets, please see our response to Q10/Q11.We constrain the correspondence between OSM object types and geometry types: a Point must come from a node, while a Polygon or MultiPolygon must come from a way or relation. Here, node, way, and relation are parameters in the PBF data.
All coordinates must be finite numbers, with longitude in [−180,180] and latitude in [−90,90]; Boolean values cannot be treated as numbers. Each polygon ring must contain at least four coordinate positions, be closed, and contain at least three distinct vertices. Cases that cannot be parsed, have an ambiguous spherical orientation, or have no identifiable interior point are rejected.
Area targets must also satisfy all three quantitative conditions: