- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes HESTIA, a hierarchical urban representation pretraining framework that organizes large-scale geospatial data into an entity–block–region–city hierarchy. It introduces an Urban Fabric Tokenizer and an Urban Interaction Transformer (UiT) with three-stage attention, trained through local function and regional fabric reconstruction. Experiments on nationwide U.S. geospatial data demonstrate strong performance across multiple spatial scales.
Critical strengths
S1: The framework is pretrained on over 263 million entities and 8.18 million blocks, demonstrating scalability.
S2: Extensive comparisons with 21 baselines, including AlphaEarth and S2Vec, show strong performance across entity-, block-, and region-level tasks.
S3: The hierarchical tokenization and attention design achieves competitive performance with only 12.9M parameters.
Critical weaknesses
Limited methodological novelty: The hierarchical attention architecture largely combines existing Transformer mechanisms, category aggregation, positional encoding, and masked reconstruction objectives. The technical contributions are primarily architectural integration rather than fundamentally new learning mechanisms.
Insufficient isolation of architectural contributions: Although component ablations are provided, the paper lacks a direct comparison with simpler hierarchical pooling or hierarchical Transformer baselines under matched inputs and training objectives, making it difficult to attribute the improvements specifically to UiT.
Limited geographic generalization: Pretraining and evaluation focus exclusively on U.S. geospatial data. Generalization across countries with different urban structures and data distributions remains unverified.
Information loss during tokenization: Aggregating entities of the same category into a single token discards fine-grained spatial arrangements and individual entity relationships, potentially limiting representation expressiveness.
Questions for authors
See the above
8 comments
Rebuttal to Weaknesses 4, Part 1
[Weakness 4, Part 1: Individual-Entity Tokenization and Region-Level Evaluation]
This is one of the issues we reflected on most deeply during this review and one we especially wanted to address carefully. The original design aggregates entities of the same category within a block into one token, so it no longer retains the independent coordinates of each entity or the relationships among individual entities. This choice does not reflect a belief that fine-grained spatial relationships are unimportant. It is a trade-off between computational scalability at the national scale and spatial detail. After aggregation, the model still retains the entity category, count or coverage area, block area, and the relative positions of blocks and regions. It can therefore still model local functional composition and spatial structure across blocks. More importantly, computation grows primarily with the number of categories actually present in each block, not with the raw entities, which reduces the computational overhead of nationwide pretraining.
The reviewer's comment also prompted us to ask a further question: what happens if entities of the same category are no longer aggregated and the model instead performs attention directly among individual entities? Can the model retain the spatial relationships removed by aggregation? Frankly, this was also the direction we were most eager to explore after reading the review. Over the past ten days, we trained and carefully checked multiple complete pretraining variants. Once the tokens changed from aggregated categories to individual entities, the sequences became substantially longer, increasing both memory consumption and training time. Each complete variant required approximately 8--10 hours. This was the most resource-intensive addition in this review cycle and the main reason our response took longer than expected. Fortunately, the effort was worthwhile. It clarified both the limits of category aggregation and the settings in which individual-entity modeling is most valuable.
In the new design, each entity is represented by an individual entity token encoding its category, relative coordinates and coverage area. All entity tokens use two-dimensional positional encoding at the entity level. The central change from the original design is that attention now operates on individual entity tokens instead of aggregated entity-category tokens. The model can therefore learn relationships directly among entities, not only interactions among categories. For clarity, we refer to the new design as individual-entity tokens and the original design in the paper as entity-category tokens.
We first examined whether the finer-grained individual-entity input also improves representations at aggregated spatial levels. For a fair comparison, the entity-category-token and individual-entity-token variants use the same downstream tasks, seven spatial folds, and MLP probe. The results for the original method, AlphaEarth, and S2Vec are taken directly from the formal evaluation in the paper. Land use uses Macro-AUPRC, Age distribution uses KL divergence, and all other tasks use . Lower is better for Age distribution, while higher is better for all other tasks. The region-level results are as follows:
| Region-level task | Entity-category tokens (original method) | Individual-entity tokens | AlphaEarth | S2Vec |
|---|---|---|---|---|
| Land use ↑ | 0.5545 0.0199 | 0.5590 0.0173 | 0.4650 0.0234 | 0.4872 0.0242 |
| Road density ↑ | 0.8956 0.0253 | 0.8921 0.0197 | 0.8556 0.0252 | 0.7726 0.0296 |
| Population density ↑ | 0.7431 0.0742 | 0.7736 0.0549 | 0.5765 0.1121 | 0.5340 0.0800 |
| Age distribution ↓ | 0.0662 0.0073 | 0.0671 0.0073 | 0.0958 0.0152 | 0.0742 0.0069 |
| Nightlight ↑ | 0.8356 0.0235 | 0.8071 0.0357 | 0.7469 0.0859 | 0.7401 0.0677 |
| Household income ↑ | 0.6023 0.0568 | 0.5951 0.0545 | 0.1202 0.2609 | 0.5286 0.0475 |
| CO$_2$ ↑ | 0.2988 0.0873 | 0.3093 0.1264 | 0.2117 0.1803 | 0.2613 0.1228 |
| Urban heat ↑ | 0.5635 0.0583 | 0.5471 0.0532 | 0.5612 0.0931 | 0.3778 0.0699 |
Part 2 continues this comparison at the block and entity levels.
Rebuttal to Weaknesses 4, Part 2
[Weakness 4, Part 2: Block- and Entity-Level Evaluation]
Continuing the same comparison and evaluation protocol described in Part 1, we evaluated the block-level representations:
| Block-level task | Entity-category tokens (original method) | Individual-entity tokens | AlphaEarth | S2Vec |
|---|---|---|---|---|
| Land use ↑ | 0.4135 0.0174 | 0.4161 0.0167 | 0.2622 0.0143 | 0.1939 0.0108 |
| Population log1p ↑ | 0.6205 0.0206 | 0.6093 0.0211 | 0.4059 0.0397 | 0.2399 0.0543 |
These results do not support the simple conclusion that finer tokens always perform better. At the region and block levels, individual-entity tokens perform better on three of the eight region-level tasks and slightly better on one of the two block-level tasks. Entity-category tokens perform better on the remaining tasks. However, the better result of the two HESTIA tokenizations exceeds both AlphaEarth and S2Vec on all ten region-level and block-level tasks. Although the outcome is not uniform, it clarifies the respective roles of the two designs. Adding object-level detail does not automatically improve tasks that depend mainly on the aggregate composition of a region.
This is not, however, the most important part of the experiment. The information loss raised by the reviewer occurs directly at the entity level. We therefore added a unified entity-category completion experiment that directly tests whether the model can use fine-grained entity context to recover a masked category. The two models use the same samples, seven spatial folds, and MLP probe. The category information of the target entity is removed before encoding. The results are as follows:
| Entity-level representation | Top-1 (%) ↑ | Top-5 (%) ↑ |
|---|---|---|
| Entity-category tokens (paper checkpoint, reevaluated under the same protocol) | 32.7 0.8 | 63.5 0.8 |
| Individual-entity tokens | 39.1 0.9 | 68.6 1.0 |
Under this unified protocol, individual-entity tokens improve Top-1 and Top-5 by 6.4 and 5.1 percentage points, respectively. This shows that the most direct value of individual-entity input appears precisely in tasks that genuinely depend on object-level information. It does not cause every aggregated-scale task to improve simultaneously. In other words, entity-category tokens still provide an efficient representation suitable for nationwide pretraining, while individual-entity tokens further enable the model to express entity-level spatial relationships.
Frankly, after completing these experiments, we were convinced by the individual-entity-token direction suggested by the reviewer. It not only produces clearer gains at the entity level but also reduces predefined category aggregation, giving the model an opportunity to learn structure directly from raw entities and their spatial relationships. This observation also echoes The Bitter Lesson: scalable learning methods should be allowed to discover structure directly from data and computation without excessive hand-designed abstractions. In the revised manuscript, we will therefore use individual entities as the objects of entity-level attention and fully report the experiments above, their computational cost, and the conditions under which each design is most appropriate. The reviewer's comment did more than prompt an additional experiment. It changed how we understand the boundary of the method and its next direction, making the paper more complete.
We once again thank you for your constructive feedback.
Response Overview
We thank the reviewer for recognizing HESTIA's data scale, geographic coverage, and overall performance. To be honest, we initially felt sad and discouraged when we first read the review. As we worked through each comment, however, we gradually realized that these questions were not simply a dismissal of our work. We must say that you gave the paper a renewed sense of purpose.
Over the past ten days, we invested substantial time in additional experiments and careful result verification. The most important and time-consuming addition was to use unaggregated individual entities as the inputs to entity attention, replacing the exclusive use of the Entity-category tokens in the original paper. We also completed experiments using H3 hexagons as alternative spatial units and cross-country generalization evaluations in Canada and Mexico with the frozen U.S.-trained checkpoint. This work required downloading large-scale map data, map projection, spatial indexing, label alignment, and careful alignment with AlphaEarth embeddings. Completing the full evaluation therefore required substantial time for implementation, computation, and verification. We sincerely apologize for keeping you waiting and greatly appreciate your patience while we completed this work. All of these experiments have now been completed and incorporated into a unified evaluation, and we will release the corresponding benchmark to support future research in the urban computing community. On this basis, we compared against S2Vec and AlphaEarth using the same spatial folds and probe protocol.
Below, we respond to the weaknesses in reverse order. First, for Weakness 4, we discuss the loss of fine-grained spatial information caused by tokenization and compare Entity-category tokens with individual-entity tokens at the region, block, and entity levels. Second, for Weakness 3, we evaluate cross-country and cross-unit generalization by applying the frozen U.S.-trained checkpoint in Canada and Mexico and by replacing Census units with H3 hexagons. Third, for Weakness 2, we add Hierarchical pooling and Hierarchical Transformer controls using the same reconstruction objectives and downstream protocol. Finally, for Weakness 1, we clarify that HESTIA contributes a unified and scalable framework for hierarchical urban pretraining.