- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces the EgoT2P benchmark allowing for the text-to-point cloud localization. Additionally, the benchmark is extended by the introduced FoRLoc method designed for the setup. The method outperforms the baselines on the introduced benchmark.
Critical strengths
- Well identified issue in the text to localization domain where the localization relativity poses a significant problem
- Simple and effective solution to the presented problem
Critical weaknesses
- The introduced benchmark is claimed as one of the novelties, also the motivation for this research comes from the relativity of the text prompt to an actual location. The design here is rather simple and only building an extra layer on the top of an existing dataset, which is rather an engineering step. It is not a weakness on its own - the most prominent issue lies in a conceptual naive approach here - the definition of “left” “right” “behind” “facing” is quite a simple design, and it is quite fuzzy definition on its own. Why has it been decided (Fig B1) that “left” is just approx. 33%? The left can mean 50% of the diagram. There is little literature supporting it (linguistic, behavioral studies?), rather ad hoc created labels to simplify the problem.
- Also, even though the benchmark claims point clouds it actually operates only in the BEV on 2D images. Another concern pertains to the experimental setup. What is the performance of the introduced method on the other benchmarks already introduced in the community. The reproduced baselines indeed have a lower performance on the introduced dataset, but how does it compare to the already established ones? This leaves the generalisation question open. Since the claim in the novelty is both on the method and benchmark - this does not seem to be substantiated.
Questions for authors
- As in the above-mentioned weaknesses, I would ask to compare the introduced method on at least one (or two) relevant benchmarks to showcase the capability of the method.
- And also, I would like to ask about conceptual clarification on the design of the benchmark (potentially with numerical evaluation wrt other benchmarks - essentially why do we need this benchmark?) Or maybe, is this really scalable conceptually (hard labels on left/right/behind)? I would like to ask more conceptual discussion here.
5 comments
Responses to Reviewer 2buW (2/3)
2. Q2 & W1 Motivation and design of EgoT2P
2.1 Why Do We Need EgoT2P?
Answer: Existing T2P benchmarks primarily focus on map-frame localization, without directly evaluating the challenges introduced by observer-relative spatial descriptions. Unlike map-frame queries, which directly constrain target positions through grounded objects and map-frame directions, egocentric queries additionally require inferring the observer's reference direction to interpret spatial relations in the map.
| T2P Benchmark | Main Focus of the Original Evaluation | Paired Map/Ego Evaluation |
|---|---|---|
| KITTI360Pose [1] | Text-guided submap retrieval, object matching, and fine localization | No |
| CityLoc-K / CityLoc-C [2] | Localization with larger local maps, multiple point-cloud sources, and complex spatial structures | No |
| Text2Loc++ evaluation datasets [3] | Localization across different levels of linguistic complexity and diverse environments | No |
| EgoT2P | Controlled evaluation of reference-frame differences in T2P localization | Yes |
EgoT2P addresses this evaluation gap through paired Map/Ego queries with matched scene evidence, including the same map, target, referenced objects, correspondences, and mention order (Appendix B.5). This enables a controlled comparison of how different spatial reference frames affect localization.
[1] Text2Pos: Text-to-Point-Cloud Cross-Modal Localization. CVPR, 2022.
[2] VLM-Loc: Localization in Point Cloud Maps via Vision-Language Models. CVPR, 2026.
[3] Text2Loc++: Generalizing 3D Point Cloud Localization from Natural Language. TPAMI, 2026.
2.2 Interpreting directional terms
Answer: We agree that directional terms such as Left/Right do not have uniquely fixed angular boundaries in natural language, and interpreting Left as the entire left half-plane is a reasonable convention. However, two distinct issues should be distinguished: the context-dependent semantics of directional language and the explicit geometric rules required for consistent benchmark evaluation. EgoT2P addresses the latter, rather than attempting to define universal human directional semantics.
Specifically, EgoT2P's query construction is grounded in prior spatial cognition, language, and behavioral research:
-
Zhang et al. (ICLR'25, §3.4) [1] discuss why human acceptance regions can depart from disjoint 90° quadrants or uniform 180° half-planes. They evaluate directional understanding using both a binary half-plane function and a continuous cosine-based function.
-
Landau (CDPS'25) [2] describes Left, Right, and Behind in terms of orthogonal axes centered on a reference object and relative position vectors between objects. This motivates our use of the facing anchor to establish a direction and of relative object positions to define the regions.
-
Furthermore, Zhang et al. (ICLR’25) cite behavioral experiments by Franklin et al. (M&C’95) [3], who found mean conceptual widths of 124°, 110°, 91°, and 92° for Front, Back, Left, and Right in Experiment 1. These widths sum to 417°, exceeding 360° and indicating overlapping directional regions. In Experiment 3, participants used Left and Right over wider angular ranges of 144° and 140° when describing positions.
These studies provide a conceptual basis for our use of reference axes and potentially overlapping directional regions. The specific angular boundaries operationalize these principles for consistent and reproducible Map/Ego evaluation, with the goal of studying Spatial Frame-of-Reference Mismatch rather than defining universal directional semantics.
[1] Do vision-language models represent space and how? evaluating spatial frame of reference under ambiguities. ICLR, 2025.
[2] Geometry and Force Dynamics in Simple Spatial Terms: Two Theories, One Resolution. CDPS, 2025.
[3] Parsing surrounding space into regions. Mem. Cognit., 1995.
Responses to Reviewer 2buW (3/3)
2.3 The “left” region in Fig. B1
Answer: The studies discussed in §2.2 motivate allowing directional regions to overlap. In Fig. B1, a support object on the speaker’s left can be a candidate for “left” even if it is also somewhat ahead of or behind the speaker. To make this rule reproducible, we allow its forward or backward offset to be up to twice its leftward offset. This gives the roughly one-third candidate region (127°) shown in the figure, including its overlap with “behind.” To address your concern about whether the observed Map/Ego gap depends on this boundary choice, we repeated the comparison using the four 90° sectors of the map-frame convention in both frames.
Unified 90° Directional Encoding. We used identical four-way sectors and boundary rules in both frames, relative to the map axes for map-frame queries and to the speaker’s facing direction for egocentric queries. We kept the other pairing conditions fixed and retrained both baselines.
| Method | Directional encoding | Map-frame R@5 ↑ | Egocentric R@5 ↑ | Δ Drop |
|---|---|---|---|---|
| VLM-Loc | Original | 59.61% | 26.01% | 33.6% |
| VLM-Loc | Unified 90° | 45.93% | 30.70% | 15.23% |
| Text2Loc | Original | 18.36% | 12.64% | 5.72% |
| Text2Loc | Unified 90° | 16.55% | 11.81% | 4.74% |
Both conventions yield the same qualitative finding. Under either convention, both models achieve lower R@5 m when the same targets are described from the speaker’s perspective. The shared 90° partition narrows the gap, especially for VLM-Loc, but the gap persists for both models.
180° half-plane interpretation. Following your suggestion, we also examine an alternative interpretation where “left” and “right” each cover a 180° half-plane, while keeping the query text unchanged. We then measure the feasible area satisfying all spatial relations in each query.
| Metric | Result |
|---|---|
| Analyzed test queries | 3,092 |
| Queries with strictly enlarged feasible regions | 99.94% |
| Median increase in feasible-region area | +62.81 m² |
As shown in the table above, this alternative interpretation allows more positions to satisfy the same query, reducing positional discrimination without necessarily better reflecting natural directional language.
2.4 Scalability Beyond Hard Labels
Answer: EgoT2P's directional labels provide a consistent way to represent observer-relative spatial relations, rather than defining the scope of the benchmark itself. The central evaluation concerns reference-frame differences, not a particular angular partition. As shown in the previous section, the Map/Ego localization gap persists under an alternative 90° directional convention.
To further examine flexibility in query composition, we evaluate 200 user-authored descriptions in which participants independently select 2–5 referenced objects and their spatial relations, as detailed in our response to Reviewer 7ezL (Q3). FoRLoc outperforms VLM-Loc across all three localization thresholds on user-authored queries. These results suggest that FoRLoc remains competitive even beyond the fixed six-object template, when users independently select referenced objects and spatial relations.
Responses to Reviewer 2buW (1/3)
We sincerely thank the reviewer for their time and thoughtful feedback. We appreciate their constructive comments, which help us better clarify the motivation and contributions of our work. We address the specific concerns below.
1.1 Performance on other benchmarks
Answer: We appreciate the suggestion to evaluate FoRLoc beyond the main EgoT2P setting. Table 4 of the original submission reports results on the native cells of KITTI360Pose. We retain the original submaps, construct egocentric descriptions for the retained targets, and train all methods on the same split. This comparison evaluates fine localization in a different local map configuration within KITTI-360.
To also examine scenes outside KITTI-360, we evaluated FoRLoc on Toronto-3D [1] and Soufflot03 from Paris-CARLA-3D [2]. We retain the maps, target positions, and referenced objects, and construct egocentric queries using the EgoT2P protocol. For each target, all methods receive the same query and target-containing submap.
FoRLoc has higher recall than the two baselines at all three thresholds in these comparisons. The additional scenes provide evidence that its benefit carries over to other point-cloud maps when evaluated with the same egocentric query protocol.
[1] Toronto-3D: A large-scale mobile LiDAR dataset for semantic segmentation of urban roadways. CVPRW, 2020.
[2] Paris-CARLA-3D: A real and synthetic outdoor point cloud dataset for challenging tasks in 3D mapping. Remote Sensing, 2021.
1.2 Methodological Contribution of FoRLoc
Answer: In the map-frame setting used by prior T2P benchmarks, “I am east of the road” tells us which side of the road to search once we identify it. Now consider “the road is on my left.” Even with the correct road identified, a north-facing speaker could be east of it, while a south-facing speaker could be west of it.
FoRLoc addresses this ambiguity by estimating the speaker’s reference state from the full query and local map. It predicts the facing anchor and the coarse target-to-anchor bearing, then uses this reference to interpret “on my left.”
Table 5 of the original submission shows that adding the predicted reference state improves localization even when the same predicted object correspondences are available. In our additional comparison for Reviewer Y1uh (Q2), predicting the target-to-anchor bearing also yields higher recall at all three thresholds than predicting alternative coarse geometric intermediate representations. Together, these results further highlight FoRLoc's advantages in egocentric localization through explicit reference-state modeling.
1.3 Role of the BEV representation
Answer: As defined in §3.1, our task is to localize a target on the ground plane of a given 3D point-cloud map. EgoT2P is built from such maps. Following VLM-Loc [1], FoRLoc uses both a BEV image and a graph of the map’s semantic instances for fine localization (§3.3). Even in this view, the model still needs to infer the observer’s reference direction to interpret egocentric relations in map coordinates. This is the spatial frame-of-reference mismatch we study.
[1] VLM-Loc: Localization in Point Cloud Maps via Vision-Language Models. CVPR, 2026.