acceptodds
Under review as a conference paper at ICLR 2027

GeoPOI: Scalable Post-Training for Selective Geographic Contextualization in ASR

Abstract

Recognizing point-of-interest (POI) names requires geographic knowledge, but irrelevant context can introduce transcription errors. We present GeoPOI, a framework for selective geographic contextualization with a frozen ASR backbone. Cell-specific adapters learn regional knowledge through difficulty-prioritized, geographically sharded post-training. A staged pipeline uses predicted entity spans for acoustic retrieval, teacher-anchored ranking to select relevant names, and trie-based logit biasing to guide transcription. Audio-conditioned routing selects a single cell, a GPS neighborhood, or the unmodified backbone. Training draws on over ten million utterances covering approximately four million POIs. On the evaluation set, with geographic gating enabled, POI character error rate (CER) falls from 14.07% to 5.24% (63% relative reduction). Component comparisons also show a 29% relative reduction in overall CER and increased all-target name containment from 47.8% to 83.0%, with general-speech CER essentially unchanged (4.92% versus 4.86%). We will publicly release our code, trained model weights, and evaluation dataset.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
speech recognition, contextual biasing, point-of-interest recognition, geographic memory, low-rank adaptation, evaluation benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HXSFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents GeoPOI, a framework for geographically contextualized ASR built around a frozen Qwen3-ASR backbone. Given speech and observed geographic context, an audio-conditioned router selects a single geographic cell, a local 3×33 \times 3 neighborhood, or no contextual intervention. Within the selected region, the system uses span-aware acoustic retrieval and ranking to select POI candidates, while regional adapters modify decoder hidden states and trie-based biasing adjusts token logits. On the evaluation set, the combined system reduces whole-utterance CER from 8.33% to 5.93% and POI CER from 17.03% to 8.94%, while general-speech CER remains nearly unchanged.

Critical strengths

The paper addresses a practical problem in contextual ASR: geographic information can substantially improve POI recognition. The proposed routing and candidate-selection pipeline is designed to control this trade-off rather than always applying geographic biasing. The empirical results are also strong. On a reasonably large evaluation set, the combined system substantially reduces both overall CER and POI CER, while general-speech CER remains almost unchanged, suggesting that the gains are not simply obtained by sacrificing non-entity recognition.

Critical weaknesses
  1. The task formulation is fairly narrow and tied closely to the observed user geography gg. The system defines a route R(x,g)R(x,g) from the current utterance xx and observed geographic context gg, and restricts retrieval to the corresponding POI pool C(R)\mathcal{C}(R). This makes sense for navigation or local-search queries, but it does not model broader conversational context. In realistic conversations, speakers may refer to places they visited previously, places they intend to visit, or places associated with another speaker, none of which need to be related to the current GPS location. The paper includes stress contexts with altered geography, but these mainly test robustness to incorrect context rather than whether the system can infer the relevant geography from dialogue history. As a result, it is unclear how well the proposed formulation generalizes beyond location-centric ASR scenarios.

  2. The technical contribution appears somewhat limited, and a central routing component is under-specified. At a high level, the method combines two familiar contextualization mechanisms: a regional adapter that modifies decoder hidden states ht→ht′h_t \rightarrow h_t', and trie-based candidate biasing that adjusts output token scores, which are essentially applications of deep and shallow fusion. The span detector, bounded retrieval, ranking, and geographic sharding are useful engineering components, but they seem largely specialized to this geo-entity setting rather than constituting a fundamentally new modeling approach. More importantly, the three-state router depends on cell logits lcl_c and a NONE logit l0l_0, through quantities such as papplyp_{\text{apply}}, l(1)−l0l_{(1)}-l_0, and l(1)−l(2)l_{(1)}-l_{(2)}. However, the paper does not adequately explain how these logits are produced or how the router itself is trained. Since this routing decision determines whether the system uses one cell, the GPS 3×33 \times 3 neighborhood, or no geographic context at all, the missing detail makes an important part of the method difficult to assess and reproduce.

Questions for authors
  1. How are the routing logits lcl_c and l0l_0 computed and trained? The paper explains how they are used for routing, but not clearly how the router itself is learned.

  2. How is the observed geography gg related to the target POI in the evaluation set? In particular, how often is the correct POI actually contained in the routed pool C(R)\mathcal{C}(R)? Since the paper notes that the test geographic contexts are constructed, this seems important for judging how realistic the setting is.

  3. Did the authors evaluate cases where the relevant POI is not tied to the speaker’s current location, such as a place mentioned earlier in the conversation, a place the speaker plans to visit, or a place associated with another speaker?

4 comments
AuthorsOpenReview ↗

The trade-off between the accuracy and universality of near-field POI correction

Thank you for recognizing the practical value and recognition gains of GeoPOI. We agree that its scope, contribution, and routing mechanism need clearer explanation.

GeoPOI targets AI glasses, automotive systems, and mobile assistants. Chinese homophones and names mixing abbreviations, Latin-script text, and numbers make POI recognition difficult. Its contribution is the coordinated treatment of geographic scope, acoustic disambiguation, scalable regional learning, and preservation of general ASR, using established adaptation and decoding mechanisms around a frozen backbone.

Our data construction benefits from high-quality POI resources available through a large commercial social platform, including aggregate regional popularity signals and common request patterns. POI information is verifiable through official sources; expression patterns are cleaned, generalized, and stripped of sensitive user information. These resources support broad coverage of everyday POI formulations without establishing unrestricted conversational generalization.

  1. Router computation and training. The added router learns whether geographic context is relevant and how broadly to search, using frozen audio features before transcription. In the evaluated implementation, it selects the central cell, the GPS 3×3 neighborhood, or bypasses local retrieval. These are retrieval actions; expressions such as “附近” (“nearby”) and “旁边” (“next to”) illustrate cues in the training data, rather than defining the supported request types. The head projects 2,048-dimensional features to 192 dimensions, applies two temporal blocks and temporal pooling, then produces routing logits through one three-class output head, with approximately 1.09 million trainable parameters. It is trained on tens of thousands of synthetic utterances derived from cleaned POI information and generalized request patterns. Labels reflect the intended target and expressed scope, including negation, background locations, and shifts of focus. Voice-balanced cross-entropy aims to reduce reliance on voice-specific correlations. Checkpoints and thresholds are selected on development data only. Uncertain scope predictions default to the central cell; the intervention gate separately controls whether enhancement is permitted. This is the scope predictor used in our additional experiment; it is distinct from the cell/NONE scoring formulation in the original description. The original component table uses fixed 3×3 geography. The added experiment evaluates the updated candidate-enhancement pipeline, not a reproduction of the original regional-adapter-plus-trie configuration. On the same 1,000 offline requests (850 synthetic POI utterances and 150 human-recorded general-speech utterances), routing reduces POI CER from 6.9452% to 6.3882% versus fixed 3×3 retrieval, an 8.02% relative reduction. Whole-utterance CER falls from 3.0695% to 2.9259%, and target-name containment rises from 75.65% to 78.82%. General-speech CER stays at 4.8873%, with identical transcripts for all 150 recordings. All requests are retained without retrieval-success filtering.
  2. Geographic context and pool coverage. Our internal usage analysis suggests that about 95% of nearby POI scenarios fall within roughly 30 km. This motivates bounded retrieval; it does not equate a cell or 3×3 neighborhood with a 30 km radius. The additional benchmark places 510 targets in the central cell, 212 in neighboring cells within 3×3, and 128 outside that neighborhood. Thus, the fixed pool contains 722 of 850 targets. Learned routing retains 721 of these 722 locally available targets (99.86%); overall pool coverage is 721/850, including the deliberately nonlocal cases. The 99.86% figure is conditional on local availability. These are controlled geographic allocations, not natural query–GPS frequencies. We will distinguish pool coverage from shortlist and top-10 recall.
  3. Places unrelated to current location. The 128 nonlocal cases test behavior when the target falls outside the local pool. They do not establish that the system can infer relevant geography from dialogue history. Our current evaluation uses a single utterance and supplied geographic context. References to earlier visits, future destinations, or another speaker’s location may require evidence beyond current GPS. We plan to incorporate application-specific dialogue context for such cases and will state this boundary explicitly. Generality across local names, regions, and expressions should be distinguished from dialogue-based geographic reasoning. The model and inference code are public. The release includes the backbone, contextual modules, and a public directory of 500 representative POIs; private nationwide catalog/geographic data and training/evaluation recordings are excluded.

Dear Reviewer,

Thank you again for your time and thoughtful review. We would appreciate your feedback on our point-by-point rebuttal, particularly whether the clarifications and additional results address your main concerns.

As the discussion period is nearing its end, could you please let us know if any key concerns remain? We would be happy to provide further clarification.

Thank you for your consideration.

Thank you for the detailed response. The added description of the router and the new routing experiment addresses my concern about how the routing mechanism is implemented and trained. However, the relationship between this new three-class router and the lc/l0l_c/l_0 formulation in the original paper remains somewhat unclear, especially since the response states that the original component table uses fixed 3×33\times3 geography. This distinction should be clarified in the paper.

The additional nonlocal examples are useful, but they mainly test robustness when the target falls outside the local pool. They do not address my broader concern about applicability. The current formulation still assumes that the supplied geographic context is the relevant one, and does not infer the relevant location from the conversational context. I therefore remain unsure how well the method generalizes beyond location-centric POI queries. I think the work would fit naturally in a speech-centric venue, while I remain less convinced about its broader applicability for ICLR.

As a separate procedural note, the repository links included in the response do not appear to be anonymized. You may want to double-check the ICLR anonymity requirements for links shared during the review period.

Firstly, tks for your advice and i have changed my repository url.

And,Recognizing proper names, including POI names, from speech remains a challenging problem without a perfect solution in current industry practice. Context is necessary to resolve ambiguities such as homophones. Compared with some existing contextual biasing and hotword injection approaches, our method substantially improves generalization.

The current POI retrieval scope is sufficient to address 90% of the problems encountered in everyday ASR use. As you correctly point out, it cannot resolve certain recognition errors involving distant POIs. We believe that addressing these cases would require more resource-intensive training of the base model and more fine-grained attention mechanisms. Our current approach represents a practical trade-off between recognition performance and computational cost.

The router is also designed to improve generalization. In the current system, it relies primarily on GPS context and semantic information from the audio. Its interaction with the ASR model is central to our design: the routing decision determines the geographic scope for retrieving POI candidates, which are subsequently used to correct recognition errors. The paper does not investigate how to optimize routing for multi-turn dialogue or more complex contextual information. Extending the router to these settings is part of our ongoing work.

Reviewer_3WkmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents an approach for supplying contextual information in the form of Point of interest (POI) names to an Automatic Speech Recognition (ASR) system. This is done in 3 ways: a) adding a regional adapter on top of a baseline LLM, b) identifying entities based on the acoustic input (via learning) and supplying them for logit biasing during decoding or c) both. For regional adaptation, the paper uses a confidence based router i.e. a specific cell, a neighborhood or none. The paper finds that both regional adaptation and biasing strategies reduce general and POI Character Error Rate relative to a baseline without any contextual adaption. Though the general idea in this paper is important, it is very hard to follow. Specifically, the paper focuses on details without giving the big picture.

Critical strengths
  • Presents two strategies for supplying context to an ASR system.
  • Demonstrates that the strategies improve Character Error Rates independently and in combination
Critical weaknesses
  • The paper focuses on minute details and does not explain the high level concepts and overall architecture. Thus, it is hard to understand the overall approach in the paper. See 'Questions' for examples.
  • The paper has multiple sentences which do not relate to the concepts introduced in the previous sentence/paragraphs.
  • The paper does not have a summary at the end.
Questions for authors
  • L014: What is 'cell-specific'? Cells have not been introduced at this point
  • L016: What is teacher-anchored ranking? Same comment for L084.
  • L022: What is 'all-name target containment'?
  • L038: What is a retrieved name? Clarify what it means for a retrieved name to fall below a budget.
  • L045: What do span conditioned queries mean? How do they differ from global queries
  • L048: What do both branches mean?
  • L052: What does preservation mean? Ditto for L116
  • L053: What are previously correct transcripts?
  • L083: The contributions are not clear. Specifically, these terms are not explained at this point: 'difficulty-prioritized coverage', 'shared negative assignments'.
  • L127: 'these are not evidence of naturally collected query-location distributions'. The phrase is unclear.
  • L135: 'Unsupported regional modules do not make lexical retrieval impossible; the candidate path remains a separate option'. Please clarify.
  • L184: 'Teacher-forced regional training restricts the adapter' - how is this done? What system is this in the Table 2 : Regional+candidate? Please provide details.
  • Section 4.3 is very hard to follow. What do scheduled visits mean? What is a schedule? It would be good to give a high level overview before describing the details.
  • L210: What are connected intervals? Please provide an example.
  • L238: What are the four semantic tasks?
  • L245: What are reader states?
  • L255: What is a decoder surface?
  • L262: Is this training distinct from the regional adapter training? Or is this only for entity prediction? i.e. for predicting membership/start/end? please clarify.
  • Section 4.6 is hard to follow.
  • L402: How is 'All-target name containment' defined?
2 comments
AuthorsOpenReview ↗

We have made the final model and the inference code publicly available

Thank you. We will clarify the workflow and terms, improve transitions, and strengthen the existing conclusion.

  1. L014: A cell is a Web-Mercator level-13 geographic grid unit. “Cell-specific” denotes its regional parameters.

  2. L016/L084: The teacher is a frozen scorer. The new ranker initially matches its candidate scores, then repairs teacher misses while retaining successful ranking margins.

  3. L022: This is the percentage of POI-bearing utterances whose final transcript contains every normalized annotated target name.

  4. L038: Retrieved names are acoustically shortlisted catalog entries: up to 256 (512 for large geographic pools). Only the top ten bias decoding in Table 2; lower-ranked names cannot.

  5. L045: Span-conditioned queries pool audio around predicted POI intervals; global queries use utterance-wide evidence. Trainable global queries and a frozen fallback remain available without predicted spans. All share one shortlist budget.

  6. L048: Regional adaptation adds cell-dependent residuals to decoder hidden states; candidate biasing adds token-logit rewards through a prefix trie. The branches share the frozen backbone and can be enabled independently.

  7. L052/L116: ASR preservation avoids regressions, assessed with non-entity and general-speech CER and correct-to-wrong changes. Ranker preservation retains successful teacher margins. This is empirical, not a per-transcript guarantee.

  8. L053: Previously correct transcripts exactly match the whole normalized reference under ASR-only (ASR-to-candidate) or regional-only (regional-to-combined).

  9. L083: Difficulty-prioritized coverage gives hard cell–name pairs identified by a fixed probe more exposure, while retaining recognized names as anchors. Shared negative assignments fix incorrect contexts and general-speech examples globally before sharding, not per shard.

  10. L127: Benchmark geography is constructed. It assesses specified query–geography relationships but does not establish the natural distribution of observed user queries and GPS locations.

  11. L135: Catalog and regional-memory coverage differ. Names remain retrievable for candidate biasing without a trained cell adapter.

  12. L184: Teacher forcing supplies reference token histories; a token mask limits adapter activation to annotated entity-prediction positions. Both Regional only and Regional + candidate use the adapter in Table 2; the latter adds candidate biasing. This training mask differs from audio-frame spans.

  13. §4.3: A schedule lists planned cell-associated presentations; a visit is one presentation, not a new recording. We probe difficulty, allocate presentations, assign negatives globally, form disjoint shards, train for four passes, then assemble. Assembly copies cell-owned parameters and averages shared projections/gates; joint-training equivalence is not assumed.

  14. L210: Consecutive frames above the entity-membership threshold form intervals: 10–12 and 30–31 become 8–14 and 28–33 with two-frame padding. We retain at most four proposals; start/end predictions provide boundary confidence rather than splitting adjacent intervals.

  15. L238: The four tasks are purpose (intent), role (mention function), spatial relation, and referent (the mention referenced by that relation). Missing labels are masked. Membership/start/end prediction and mention detection have separate supervision.

  16. L245: Reader states are normalized 128-dimensional frame outputs of the frozen acoustic reader, used for retrieval queries and acoustic candidate scoring.

  17. L255: A decoder surface is the name text supplied to the tokenizer/decoder. Textual forms are deduplicated to save candidate slots; this does not distinguish physical POIs sharing a name.

  18. L262: Retriever training is separate from regional-adapter and span training. Heads train sequentially: span detection, retrieval from predicted spans, then ranking from retrieved candidates. The backbone, projector, reader, and catalog embeddings remain frozen.

  19. §4.6: Ranking aims to place correct names within the decoding budget using acoustic, retrieval, and semantic evidence. Training repairs teacher misses and preserves successful margins. NONE supports optional rejection; Table 2’s top-ten policy bypasses this lexical threshold.

The model and inference code are public. The release includes the backbone, contextual modules, and a public directory of 500 representative POIs; private nationwide catalog/geographic data and training/evaluation recordings are excluded.

Thanks for your clarifications. I believe that the paper needs to be rewritten to improve the flow and make it easy to follow - hence, I will keep my original scores.

Reviewer_g73bOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work adds location-aware POI to a frozen Qwen3-ASR model through adapters by 66.5K map cells which are trained in 12 geographically disjoint shards and then assembled. A pipeline comprosed of span detection, acoustic retrieval, learned ranker and top-K trie logit biasing. Training used over 10M utterances some of which is synthesized from human voice.

Authors report character error rate (CER) improves from 8.33 to 5.93 while POI CER falls from 17.03 to 8.94 and name containment rises from 47.8 to 83.0

Critical strengths

2x2 comparison is clear with explicit accounting of what it breaks -- correct-to-wrong counts, non-POI CER, general speech. It shows candidate biasing carries most of the gain, and regional adapters add a small increment. That breakdown is definitely useful for any production deployment for voice search.

Critical weaknesses
  1. Abstract claims POI CER improvement from 14.07 to 5.24, but it is never defined. I can see 5.24 matches combined POI CER on the 6615 requests, whose target already reached the top 10, which conditions on retrieval success. HOwever, 14.07 appears in no table.
  2. Router (mentioned in contribution #3) seems like it is not used in the evaluation, which fixes the GPS 3x3 neighborhood.
  3. Lack of ablation for training recipe (contribution #2).
  4. The test conditions seem to be too optimistic -- for example geographic contexts are constructed, items not in catalog are excluded, and the synthetic share of the test audio is not reported.
  5. Some important metrics are missing -- regional memory is about 4.6B FP32 parameters (19.19 GB), which is larger than the backbone, and latency is not measured / reported.
Questions for authors
  1. Could you define figures 14.07 and 5.24 ? Is 5.24 conditioned on the target reaching the top 10?
  2. Could you report test audio composition -- what share of the test audio is synthetic or voice-conditioned? Could you report Table 2 on human recordings only.
  3. How are test geographic contexts generated? How often is the target outside the 3x3 neighborhood?
  4. Could you report end-to-end results with router enabled?
1 comment

Dear Reviewer,

Thank you again for your time and thoughtful review. We would appreciate your feedback on our point-by-point rebuttal, particularly whether the clarifications and additional results address your main concerns.

As the discussion period is nearing its end, could you please let us know if any key concerns remain? We would be happy to provide further clarification.

Thank you for your consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

We have made the final model and the inference code publicly available, and have also conducted new experiments and performed calibration adjustments.

Thank you for the careful review and for recognizing the value of our component-wise evaluation.

GeoPOI targets AI glasses, automotive cockpits, and mobile devices, where Chinese homophones, abbreviations, Latin-script names, and numbers challenge POI recognition. Conventional post-training or hotword injection can degrade general ASR or miss intended names. Our router bounds retrieval; span/ranker accuracy reaches 98%; gates aim to preserve general ASR at low cost; cell-wise training and shared-basis compression address training/storage costs for approximately four million POIs. Internal data suggest 150–160 km² covers 95% of nearby POI scenarios. Richer dialogue context is future work.

The model includes the backbone and contextual modules; inference code is public. The public POI directory contains 500 representative entities; private nationwide catalog/geographic data and training/evaluation recordings are excluded. POI information is available on official websites; internally cleaned utterance formulations are generalized into common expressions without sensitive user information.

We checked the original records: 14.07%→5.24%14.07\% \rightarrow 5.24\% are POI CERs on the same 6,615-request subset where at least one annotated target enters the selected top-10, not whole-utterance or unconditional CERs. POI CER divides character errors assigned to annotated POI spans by total reference POI characters. Full-set ASR-to-combined results are 8.33%→5.93%8.33\% \rightarrow 5.93\% whole-utterance CER, 17.03%→8.94%17.03\% \rightarrow 8.94\% POI CER, and 47.76%→82.96%47.76\% \rightarrow 82.96\% all-target name containment. These will be the headline results; the subset results will be labeled diagnostic.

The original main table uses fixed GPS 3×33\times3 retrieval. We added a single-head audio-semantic router to the latest enhancement pipeline. From frozen audio features, without an ASR transcript, it selects the central cell, 3×33\times3 neighborhood, or no geographic retrieval. Uncertain predictions default to the central cell; the intervention gate is unchanged.

Offline evaluation compared ASR with enhancement disabled, fixed 3×33\times3 retrieval, and learned routing on the same 1,000 requests: 850 synthetic POI utterances and 150 human-recorded general-speech utterances. All were retained without retrieval-success filtering.

ConfigurationWhole CERPOI CERGeneral CERASR only4.6625%13.1245%4.8873%Fixed 3×33.0695%6.9452%4.8873%Learned routing2.9259%6.3882%4.8873%\begin{array}{lrrr} \text{Configuration}&\text{Whole CER}&\text{POI CER}&\text{General CER}\\ \hline \text{ASR only}&4.6625\%&13.1245\%&4.8873\%\\ \text{Fixed }3\times3&3.0695\%&6.9452\%&4.8873\%\\ \text{Learned routing}&2.9259\%&6.3882\%&4.8873\% \end{array}

Routing reduces POI CER by another 8.02% relative to fixed retrieval; target-name containment rises from 75.65% to 78.82%. All 150 general-speech transcripts remain identical. Checkpoint selection and threshold calibration used development data only. This separate candidate-enhancement experiment supplements, rather than reproduces, the original regional-adapter-plus-trie configuration.

The original evaluation audio is approximately 10% human-recorded and 90% synthetic. Synthetic queries include “去XX地方,” “XX怎么样,” and “导航去XX”; they cover names/expressions but do not establish robustness to natural acoustic variability. The new 150 human recordings assess general-speech preservation, not human-only POI reproduction of Table 2.

Our catalog covers China; sampling uses aggregate regional POI popularity from our platform. Evaluation geography is constructed, not observed user GPS. The 850 new POI requests allocate 510 targets to the central cell, 212 to neighboring cells within 3×33\times3, and 128 outside. These are controlled allocations, not natural location frequencies. We will distinguish geographic-pool, shortlist, and top-10 coverage.

The original FP32 regional bank occupies 19.19 GB. The released shared low-rank bank retains all 90,760 cell entries in approximately 3.56 GB of FP32 weights. Duplicated factors share one basis with all regional coefficients retained. We verified exact reconstruction of duplicated factors and identical outputs/decision traces on 25 paired offline requests. This establishes storage-conversion equivalence, not a controlled same-checkpoint comparison between the original and newer banks.

The full retrieval/enhancement pipeline adds approximately 100 ms. POI correction is inactive in most everyday scenarios, so we expect limited impact on the overall user experience.

Human recordings were collected under authorized arrangements, with a substantial portion from internal team members. All contributors authorized recording use. We will clarify permitted uses, including voice-reference synthesis and redistribution of recordings linked to geographic context; any audio release must remain within applicable permissions.

We will incorporate these clarifications in the manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.