acceptodds
Withdrawn from ICLR 2027

CityCode: Building Agent-Operable 3D Cities from Sparse Aerial Observations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Urban spatial intelligence calls for 3D cities that agents can measure, interpret, and modify across successive tasks. Maintaining such cities as persistent, code-based states requires lightweight acquisition and support for consistent state updates without repeated dense reconstruction. We therefore formulate a new setting for grounded urban world construction: generating large-scale, instance-level parameterized 3D cities from one low-altitude aerial RGB image per region and known camera parameters, while recovering metrically scaled buildings and explicit road geometry and connectivity. We introduce CityCode, an agent-operable representation that unifies metric geometry, urban topology, visual semantics, and 3D assets through persistent entity identities, with dependency-aware versioned execution for validated updates and historical restoration. We further develop Air2Code to construct this representation by combining visual structure extraction with adapted metric depth estimation. To address occlusion and incomplete coverage, recovered structure and observed appearance guide generative completion and subsequent structure-aware 3D generation for buildings and backgrounds. Experiments on CitySample-Aerial and UrbanScene3D demonstrate substantially more accurate metric structure recovery and superior visual fidelity to the target cities compared with the evaluated baselines. Across 50 task chains of 50 steps each, CityCode supports continued querying, editing, planning, and historical restoration over evolving city states. Together, CityCode and Air2Code turn sparse aerial observations into persistent, agent-operable urban worlds for long-horizon urban spatial intelligence.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D city generation, spatial intelligence, aerial imagery

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_46u5OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces CityCode, a representation that unifies metric geometry, urban topology, visual semantics, and 3D assets through persistent entity identities. It also presents Air2Code, a pipeline that combines visual structure extraction with adapted metric depth estimation to construct this agent-operable representation from sparse aerial observations. Experiments on CitySample-Aerial and UrbanScene3D show more accurate metric structure recovery and better visual fidelity to the target cities than the evaluated baselines.

Critical strengths

(1) The paper introduces CityCode, an agent-operable representation that links metric urban structure, observational evidence, visual semantics, and 3D assets through persistent entity identities.

(2) Air2Code provides a complete pipeline for constructing this representation, combining visual structure extraction with adapted metric depth estimation.

(3) Experiments on both synthetic and real-world-derived scenes show improved metric geometry and leading visual quality among the evaluated baselines on CS-Aerial and UrbanScene3D.

Critical weaknesses

(1) The paper would benefit from further revision and polishing. For example, the introduction does not include an explicit list of contributions, and the conclusion is brief.

(2) The main contribution appears to be a system pipeline that combines processing steps from existing work, including image completion and 3D generation. The individual components are largely drawn from prior methods.

(3) The pipeline relies mainly on existing models as its core generators: TRELLIS.2 for buildings, Pixal3D for backgrounds, and GPT-image-2 for appearance completion.

(4) Aerial RGB images provide rich appearance and geometric cues about the target city, so strong results from such input are somewhat expected.

(5) The paper also does not justify its choice of GPT-image-2, TRELLIS.2, and Pixal3D, or compare them with alternatives. This makes it unclear whether the gains come from the proposed design or from these pre-trained and proprietary models.

Questions for authors

(1) How does performance degrade with pose or intrinsic noise?

(2) What are the end-to-end runtime and API cost for the ~22.58 km² CS-Aerial area?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 46u5 (1/2)

Please read your individual response first: shared points are summarized here, with references to the General Response for details.

Thank you for the review, and for recognizing our entity-linked representation, complete pipeline and improved metric geometry. Your specific questions on pose and intrinsic noise are answered in full below. The revised paper and supplement mark all changes in blue. We start with where the gains come from.

Weaknesses (3) and (5): the core models are adapted, compared with alternatives, and not tied to proprietary models.

We respectfully clarify that the core models are not used off the shelf, and that the paper already compares them with their base versions (General Response B2, Table 4):

  • Adapted, not off the shelf. The building generator adapts TRELLIS.2 with structure-aware building priors (footprint and height-to-footprint-ratio losses), and its sampled structure is corrected to the recovered building height before geometry and materials are generated; GPT-image-2 runs inside our structure-guided appearance completion; CityCode-Depth adapts MoGe2-Aerial with metric-scale supervision and ground-anchored calibration.
  • Compared with alternatives. With the official TRELLIS.2, prism IoU drops from 0.737 to 0.662; without structure guidance in completion, to 0.671; with the unadapted, uncalibrated depth model, camera-depth RMSE rises from 6.27 to 69.58 m. The gains therefore come from our adaptations and mechanisms, not from the base models.
  • Not dependent on proprietary models. We further replace both proprietary models (GPT-image-2 and Gemini 3 Pro Image) with open-weight ones (General Response B1): FLUX.2-klein for GPT-image-2 keeps building geometry (prism IoU 0.775 vs. 0.737) with a moderate drop in appearance, and SAM 3.1 for Gemini leaves DSM RMSE unchanged (14.39 vs. 14.37 m); CityCode stays ahead of the strongest baselines in both cases.

Each model fills one role and was the strongest available at submission. Pixal3D generates only the non-building background and is conditioned on the recovered ground and elevated height fields.

Weakness (2): CityCode is not a simple combination of existing methods.

As explained in General Response A and above, our contribution comprises a new setting, a pipeline (Air2Code) with models adapted with structural priors and new mechanisms, and, to our knowledge, the first exploration of long-horizon tasks (querying, editing, planning and restoring earlier states) in an observed city. Removing the adapted models or mechanisms degrades results (General Response B2). These components can also benefit city generation beyond our pipeline: structure-and-observation-guided generation (structure-guided appearance completion followed by the structure-aware building generator) yields buildings that match both the observed appearance and the recovered footprint and height, and CityCode-Depth recovers metric depth from low-altitude aerial views.

Weakness (4): informative input.

As shown in General Response B3, the input is in fact sparse: one aerial view per region, so metric scale is not directly visible and most facades are unseen. Given exactly the same RGB and cameras, Sat3DGen and Sat2Density++ reach DSM RMSE 37.03 / 33.99 m vs. 14.37 m for CityCode, and UrbanWorld2.0, given even more input (GT heights and layouts), still trails on every visual metric. Strong results thus come from our method, not from the input alone.

Weakness (1): writing.

We agree; as listed in General Response F, Sec. 1 now has an explicit contribution list and Sec. 5 an expanded conclusion with limitations.

AuthorsOpenReview ↗

Response to Reviewer 46u5 (2/2)

Question (1): pose and intrinsic noise.

At realistic levels it does not degrade noticeably. Our test cameras already vary per patch (tilt 2–4° from nadir in any direction, flying height 260–320 m). On top of this, we add errors to each patch's known camera and re-run the whole structure stage (App. G.10):

  • Three error levels from UAV sensor specifications and calibration studies (Applanix APX-15; DJI Phantom 4 RTK; Zhou et al., 2020; Tomsett & Leyland, 2021; Pérez et al., 2011; Abeho et al., 2024): survey-grade (manufacturer specifications), field-flight (errors reported in real flights) and stress (RTK loss and poor calibration).
  • Six camera parameters (position, roll, pitch, heading, focal length, principal point), perturbed one at a time ("each"; ranges are over parameters) and all jointly at the field-flight level.

At survey-grade and field-flight error levels, building F1 is unchanged and geometry changes by at most 0.31 m.

Error level (position; roll/pitch, heading; focal; principal point)[email protected] ↑IoU ↑XY error (m) ↓Height MAE (m) ↓DSM RMSE (m) ↓
None83.4 %0.8122.033.6014.37
Survey-grade (2–3 cm; 0.025°, 0.08°; 0.05 %; 1 px), each83.4–83.7 %0.810–0.8122.04–2.073.57–3.6714.37–14.38
Field-flight (5 cm; 0.1°, 0.3°; 0.5 %; 3 px), each83.1–84.0 %0.804–0.8112.03–2.173.58–3.7214.37–14.45
Field-flight, all jointly84.1 %0.7892.343.7614.60
Stress (20 cm; 0.5°, 1°; 1 %; 10 px), each79.6–83.6 %0.746–0.8112.05–2.873.63–4.1914.37–15.18

Two properties keep the errors small:

  • ground-anchored scale calibration absorbs about 87 % of a focal-length error;
  • each building is recovered from a single patch, so errors do not accumulate across patches.

Only stress-level attitude errors cause metre-level shifts, as expected for a viewing distance of about 300 m. Under realistic camera errors, the recovered metric structure therefore stays stable.

Question (2): runtime and cost.

As detailed in General Response E, we first clarify that 22.58 km² is the BigCity area used only to train the depth model, which needs no API calls; the evaluated city covers 2.79 km² (79 regions; Sec. 4.1, revised). End to end, one 400 m region takes 32 min and USD 1.00 (0.27 GPU-h), and regions are independent, so they run in parallel. The whole evaluated city takes 801 API calls, USD 79.1 and 6.7 GPU-h.

We would be grateful to know whether these responses address your concerns, and are happy to clarify further.

Dear Reviewer 46u5, we would like to kindly follow up on our response. We would appreciate any feedback on concerns that remain and would be happy to discuss them further. Thank you for your time!

I thank the authors for their detailed response and the effort they put into the rebuttal.

However, after carefully reading the response, my main concerns remain. Despite the authors’ extensive explanations, I still find that the paper lacks novelty. Beyond combining several existing components into a single pipeline, I am unable to identify a clear technical contribution.

Several of my questions also remain unaddressed. For example, the authors argue that RGB input is not information-rich because other methods perform worse with the same input. This reasoning is flawed: it only shows that a larger combined system performs better, not that RGB provides insufficient information. The gains may stem from increased model capacity rather than the proposed design.

Overall, the authors’ response does not adequately address most of the concerns raised in my review, so I will keep my original score.

Reviewer_s14iOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents CityCode, a city representation designed for use by agents. Air2Code combines image segmentation and monocular depth with camera and ground references to recover building geometry and road structure. It then uses image completion and 3D generation models to create building and background assets and assemble them into a city. CityCode represents and maintains the resulting city. Persistent entity IDs link building geometry, road and block relations, observation evidence, and 3D assets. Validated, versioned updates support queries, edits, planning, and restoration of earlier states. So, this allows agents to keep working with the city over successive tasks. The authors evaluate reconstruction and visual quality on CS-Aerial and UrbanScene3D, and use task chains to test querying, editing, planning, and restoration.

Critical strengths
  1. CityCode and Air2Code connect city construction with later interaction in a practical and reasonable way. The paper also provides algorithmic detail to understand how the method is implemented.
  2. The evaluation is broad. It uses several metrics on two datasets to assess reconstruction and generation quality, includes component ablations, and tests continued interaction through 50 task chains of 50 steps each.
Critical weaknesses
  1. The research contribution beyond complex tool integration is unclear. Well, the overall goal is clear, but the paper does not sufficiently distinguish a broader task scope and adaptations of existing tools from new methodological ideas. The method mainly describes the assembled workflow. It should clarify whether the contribution lies in a new way of organizing the components, a new mechanism for coordinating them, or something else.
  2. Some core claims need more direct experimental support. Tables 2 and 3 do not assess road connectivity or how reconstruction errors affect queries and planning. As acknowledged in Appendix G.7, many baseline runs in Table S12 stop because the required workflows are unavailable. After all, persistent and reliable city interaction is a central motivation of the paper. A more direct evaluation would compare shared operations under matched execution budgets and check whether later queries and plans reflect edits, including after restoration. Reporting outdated query results, constraint violations, and repair effort would also help assess state maintenance beyond completion rates.
Questions for authors
  1. Please summarize the main contributions in the Introduction and clarify what is new in Air2Code and CityCode compared with the closest work on monocular building reconstruction, city generation, and editable scene representations.
  2. Given the different inputs used by the comparison methods, how should readers interpret the reported performance gains? Is CityCode-Depth mainly a domain adaptation of MoGe2-Aerial, or are there specific methodological innovations beyond fine-tuning and scale calibration?
  3. The presentation of the introduction and method sections could be polished, particularly the logical flow and the definitions of symbols. For example, the paragraph introducing CityCode and Air2Code could use clearer transitions between representation, construction, generation, and state updates. The placement of arrows and labels in Figures 1 and 4 could be improved. Some subfigures in Figures 3 and 4 are also blurry. A more carefully arranged layout can make these figures easier to read. These are mainly presentation issues, but they make parts of the paper hard to follow. If I have misunderstood any part of the paper, I would welcome clarification and further discussion.
4 comments
AuthorsOpenReview ↗

Response to Reviewer s14i

Please read your individual response first: shared points are summarized here, with references to the General Response for details.

Thank you for the detailed and constructive review, and for seeing the link between construction and later interaction as practical and reasonable. Questions specific to you, including how reconstruction errors affect queries and planning, are answered in full below. The revised paper and supplement mark all changes in blue.

Question 1 & Weakness 1 (contributions and closest work).

General Response A answers this in detail, and its Table A1 compares the closest work, as you suggested. In short, the contribution lies in three connected parts:

  • New setting (CityCode): a metric, parametric and versioned city recovered from aerial observations rather than supplied as layouts.
  • Pipeline (Air2Code): constructs CityCode from aerial images, using models adapted with structural priors and new mechanisms (e.g., ground-anchored scale calibration, constraint-checked assembly).
  • Long-horizon use: agents query, edit, plan and restore earlier states over 50 chains of 50 steps.

The revised Sec. 1 lists these contributions, and Sec. 2 discusses the three lines of work you name.

Weakness 2 (more direct experimental support). We agree and add one analysis per point you raise: (a) road connectivity, (b) how reconstruction errors affect queries and planning, (c) a comparison with the baselines on the same tasks, and (d) state maintenance after edits and restorations.

(a) Road connectivity. As shown in General Response D1, the predicted road graph reaches ground APLS 0.886, and about 90 % of shortest routes match the ground truth.

(b) Errors → queries and planning. We run the same queries with the same CityCode tools on our reconstructed city and on the ground-truth city; an answer is correct if it matches the ground-truth answer within tolerance (5 m / 5° for distance and bearing; 2.5 m for the nearest road; the same building for the tallest building; the same target and ≤ 10 % route-length error for routes). Queries involve only reconstructed buildings (App. G.12).

QueryCorrect ↑Median error ↓Main error source
Distance and bearing between two buildings88.3 %1.36 m, 0.34°building footprint
Tallest building in a region92.1 %height 3.7 mbuilding height
Nearest road to a building91.7 %1.19 mbuilding footprint
Route to the tallest building in a regiontarget found: 83.2 %–building height

Most answers are correct (median distance error about 1 m), and the remaining errors trace back to building footprint or height errors, so improving these two directly improves the queries.

(c, d) Comparison with baselines and state maintenance. General Response D2 compares this directly: the three methods run the same 5 chains with the same agent, and Table D2 reports the quantities you list (queries depending on earlier edits, restorations, constraint violations and repair effort). CityCode passes all 5 chains with no constraint violation and the least repair effort, whereas the baselines pass 2/5 and 1/5 even when the agent may retry and write missing workflows. Over the 50 reference chains (Table S14), all edit-dependent queries are also correct, every restoration reproduces the exact state, and no accepted edit violates constraints.

Question 2 (inputs; CityCode-Depth).

As shown in General Response B3, given the same or even more input, the baselines remain far behind on every metric. We agree that CityCode-Depth is a domain adaptation of MoGe2-Aerial rather than a new architecture; ground-anchored calibration further lowers camera-depth RMSE from 8.75 to 6.27 m, and instance recovery is still needed to obtain entities (Table 4a, Table 3).

Question 3 (presentation).

Thank you; we rewrote the transitions between representation, construction, generation and state updates in Sec. 1, unified the notation with a symbol table (App. B), and redrew the arrows and labels in Figs. 1 and 4. All subfigures in Figs. 3 and 4 are embedded at over 1,300 ppi, so zooming in shows full detail, and App. A adds enlarged versions of both figures (Figs. S5, S6). The remaining blur in Fig. 3 comes from some baselines' own renderings, not from image resolution; it is most visible in the blurred, stretched surface textures of Sat3DGen (Sec. 4.2).

Thank you again; we would welcome further discussion if any point remains unclear.

Dear Reviewer s14i, we would like to kindly follow up on our rebuttal. We would appreciate your thoughts when you have a chance, and are happy to clarify any remaining questions. Thank you for your time!

Thank you for the response and additional experiments. The new road-connectivity and interaction evaluations address some of my earlier concerns, and the clarification of the model adaptations is helpful.

However, my main concern remains. The work still reads primarily as an integration of existing workflows with task-specific adaptations. The ablations show that these adaptations are useful within the pipeline, but I am not convinced that they establish a sufficiently distinct methodological contribution. My concern is not the use of existing models itself, but what new insight the paper offers beyond making these components work together.

The scope also remains too broad, spanning reconstruction, generation, scene representation, and long-horizon interaction. These are related, but combining them into a complete system does not, by itself, establish a focused research contribution. I would encourage the authors to narrow the scope, identify the central research question, and organize the method and evaluation around it.

I therefore maintain my original recommendation of weak rejection.

Thank you for your thoughtful follow-up and for noting that the new road-connectivity, interaction, and adaptation analyses helped address your earlier concerns. Below we briefly clarify our central research question and core insight.

1. One focused question in grounded city generation.

Within the established direction of 3D city generation (e.g., GaussianCity, MajutsuCity, UrbanWorld2.0), prior methods rely on pre-supplied maps/layouts or synthesize ungrounded scenes, whereas observation-based methods (Sat3DGen, Sat2Density++) output monolithic surfaces without operable entities. Our work studies one central question: how to construct a visually and metrically grounded, entity-level 3D city directly from sparse observations. On this core grounded city construction task (Tables 1–2), CityCode leads on 21 of 22 geometry and visual metrics on CS-Aerial and real-world UrbanScene3D (DSM RMSE 14.37 vs. 33.99 m, and 12.87 vs. 19.88 m without real-scene fine-tuning; ranked 1st across all four VLM judges), outperforming even UrbanWorld2.0 given GT layouts and heights (Part B3).

2. Core insight beyond tool integration.

Our key insight is to use recovered metric entities as an explicit structural invariant guiding generative completion and 3D synthesis, decoupling structural fidelity from specific base models:

  • Backbone-agnostic coupling (Parts B1–B2). Replacing proprietary models with open-weight ones (SAM 3.1, FLUX.2-klein) preserves geometry (DSM RMSE 14.39 vs. 14.37 m; prism IoU 0.775 vs. 0.737), whereas breaking our entity-structure coupling sharply degrades results (prism IoU 0.737 → 0.619; camera-depth RMSE 6.27 → 69.58 m).
  • Natural payoff of parameterized structure (Part D). Because the constructed city is already parameterized into metric entities and road topology (0.886 ground APLS; 83–92 % query agreement), maintaining versioned dependencies naturally unlocks reliable long-horizon operations: passing all 5/5 50-step chains with 0 violations, whereas the same agent writing missing workflows on baselines incurs 7 and 25 violations and 4–5× runtime (passing 2/5 and 1/5 chains).

We appreciate your advice to organize the exposition more tightly around this central question and will sharpen Sec. 1 and Sec. 3 accordingly. Thank you again for the constructive discussion.

Reviewer_xVxZOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes constructing "agent-operable," instance-level parametric 3D cities starting from a single low-altitude aerial RGB image per region together with known camera intrinsics K and pose (R, t). The output jointly contains measurable building geometry (an LoD1 proxy defined by footprint, base elevation, and height), explicit road geometry and connectivity (vectorized junctions and centerlines, blocks, and their relations to buildings), and 3D assets that can be bound to entities.

Critical strengths
  1. The task setting is meaningful.
  2. Metric structure recovery is the most solid quantitative result in the paper. Table 2 reports a large improvement on CS-Aerial, and Sec. 4.3 specifically notes that this is achieved without fine-tuning on real scenes.
  3. The engineering constraints in the assembly step are explicitly modeled. Ground contact, road intrusion, and neighboring-building clearance are explicitly evaluated; assets and transforms are bound only after these checks pass. This makes it a complete system with explicit failure paths and rejection mechanisms.
Critical weaknesses
  1. The construction and split of the self-built CS-Aerial benchmark are insufficiently disclosed, raising a risk that training and evaluation share the same source. Sec. 3.2.1 states outright: "we fine-tune MoGe2-Aerial on rendered BigCity RGB–depth pairs from CS-Aerial" — that is, the depth-adaptation data comes from CS-Aerial itself. Sec. 4.1 then says "BigCity provides rendered RGB–depth pairs for adaptation. A separate map provides evaluation data. Their sampled areas cover approximately 22.58 and 2.79 km², respectively" — but it does not state whether this "separate map" and the map used for adaptation come from the same city asset pack, whether they are spatially disjoint, whether any patches overlap, or which portion each of the 22.58 and 2.79 km² figures refers to.
  2. Visual fidelity relies on a single VLM judge, which may introduce evaluation bias.
  3. Compute cost, latency, and monetary cost are not reported. The pipeline invokes at least Gemini 3 Pro Image (instance and semantic masks), GPT-image-2 (building appearance completion and BEV completion), MoGe2-Aerial (fine-tuning), TRELLIS.2 (adaptation), and Pixal3D (background patch generation). What is the cost of producing a single asset, and is any human intervention involved?
Questions for authors
  1. Please clarify: (i) which part of CS-Aerial and which evaluation map the 22.58 km² and 2.79 km² figures correspond to; (ii) whether the BigCity rendered pairs used to fine-tune MoGe2-Aerial are guaranteed to be spatially non-overlapping with the evaluation patches used in Table 1 and Table 2; (iii) whether both come from the same city asset pack of the Unreal Engine City Sample project.
  2. Is the VLM judge asked to assess visual realism or consistency with a reference view? Would using a different VLM change the results?
  3. For a single 400 × 400 m patch, what are the end-to-end latency, the number of calls to each model/API, the peak GPU memory, and the estimated monetary cost? Are the eight A100 GPUs used for training, for adaptation, or for inference? If GPT-image-2 and Gemini 3 Pro Image were replaced with open-source alternatives, how much would the metrics in Table 1 and Table 4(b) degrade?
  4. The building generator is adapted on a curated subset of 5,402 assets from SAT-SKYLINES. How should one proceed when the target scene contains building types or architectural styles that are not represented in this library? Is there a fallback path, and how does generation quality degrade in that case?
2 comments
AuthorsOpenReview ↗

Response to Reviewer xVxZ

Please read your individual response first: shared points are summarized here, with references to the General Response for details.

Thank you for the positive assessment of the task setting, the metric structure recovery and the explicit assembly constraints, and for the precise questions. Your specific questions on other VLM judges and unseen building styles are answered in full below. The revised paper and supplement mark all changes in blue.

Question 1 & Weakness 1 (data split).

As detailed in General Response C, the test map is never seen in training or model selection:

  • (i) Areas. 22.58 km² is BigCity, used only for depth adaptation and checkpoint selection on its own validation split; 2.79 km² is SmallCity, used for all CS-Aerial testing.
  • (ii) No overlap. The two are separate maps, so no evaluation patch overlaps a training patch.
  • (iii) Same asset pack. Yes, both come from UE City Sample and share its materials, as do earlier benchmarks such as MatrixCity and OpenFly; each city's layout and buildings, however, are generated separately (5.1 % exact massing matches). CityCode also leads on real UrbanScene3D cities without real-scene fine-tuning (DSM RMSE 12.87 vs. 19.88 m; Table 2).

You are right that the sentence in Sec. 3.2.1 was ambiguous; we rewrote it and added the split and map details in Sec. 4.1 and App. C.1.

Question 2 & Weakness 2 (VLM judge).

A different VLM does not change the conclusion. We re-ran the judgments with judges from other model families: Qwen3-VL-32B and GLM-5.3-Flash (the highest-ranked open-weight model on LMArena Vision, Oct 2026) on all 15,414 Table 1 pairs, and the commercial Gemini 4 Argon on a stratified 50-view subset (1,575 pairs). Each cell is CityCode's preference rate against all six Table 1 baselines, with its rank among the seven methods:

CityCode ranks first in every column under every judge.

Table 1 columnDeepSeek V4.1 Flash (paper)Qwen3-VL-32BGLM-5.3-FlashGemini 4 Argon (50-view subset)
CS-Aerial Visual93.8 % (1st)99.5 % (1st)94.7 % (1st)99.7 % (1st)
CS-Aerial GT-cond.90.0 % (1st)99.1 % (1st)83.8 % (1st)97.7 % (1st)
UrbanScene3D Visual99.6 % (1st)100.0 % (1st)99.2 % (1st)98.0 % (1st)
UrbanScene3D GT-cond.98.4 % (1st)99.9 % (1st)98.1 % (1st)100.0 % (1st)
  • The closest case is CS-Aerial GT-cond. under GLM (83.8 vs. 80.9 % for the runner-up).
  • What the judge assesses. The two criteria are separate: Visual is instructed to "ignore whether the candidate reconstructs the exact reference building" and judges plausible geometry, detail and artifacts; GT-cond. is instructed to "judge similarity to REFERENCE", geometry first, then appearance. App. G.1 gives the protocol, App. I the full prompts and App. G.9 the other judges.

Question 3 (open-source replacement).

As shown in General Response B1, geometry does not degrade and CityCode stays ahead of the baselines. Table 4(b): prism IoU 0.737 → 0.775, while PSNR drops 1.8 dB. Table 1: a full UE re-render was not feasible within the discussion period; on a 480 m downtown region rendered from the Table 1 cameras, PSNR is 0.3 dB lower (13.67 vs. 13.98 dB), LPIPS is nearly equal (0.555 vs. 0.554), and the VLM judge prefers the open-source variant. Replacing Gemini with SAM 3.1 leaves DSM RMSE at 14.39 vs. 14.37 m.

Question 3, continued, & Weakness 3 (cost, GPUs, human intervention).

As detailed in General Response E: per 400 m region (mean), 32 min, 10.1 API calls (2 Gemini, 0.8 background, 7.4 building completion), peak GPU memory 22.2 GB and USD 1.00; per building, USD 0.053 and 3.1 min. The eight A100 GPUs are used for training and inference; one-off training and adaptation take about 1,690 GPU-h. There is no manual step: an agent checks 26 multi-part detections once to confirm instance grouping.

Question 4 (unseen styles, fallback).

The generator is image-conditioned rather than retrieving from the library, so a new style does not need a matching asset:

  • Already tested. The 5,402 assets (curated from SAT-SKYLINES, over 50,000 stylized buildings) only train the generator; both test sets, City Sample and real UrbanScene3D cities, contain styles outside them, and CityCode still leads (Tables 1–2).
  • Style is handled before 3D generation. Structure extraction and appearance completion use general-purpose image models, not tied to our library. The generator, built on TRELLIS.2 pretrained on large-scale general 3D assets, then lifts the completed image; our adaptation mainly keeps its outputs building-like and consistent with the recovered footprint and height.
  • If a style is poorly represented, the mesh may look less faithful, but the building's position, footprint and height are unaffected: they are recovered before generation, and the mesh is fitted to them.

Thank you again; we welcome further questions.

Dear Reviewer xVxZ, just a brief follow-up on our response. We would be grateful to hear whether it addresses your concerns, and welcome any further questions. Thank you!

Comments on the paper

5 comments
AuthorsOpenReview ↗

General Response (1/5)

Reading guide: we suggest reading your individual response first. This General Response is longer: it covers points raised by several reviewers in detail, and the individual responses point to the relevant parts.

We thank all reviewers for their careful reading and constructive comments, and are encouraged that they found the task setting meaningful (xVxZ), the entity-linked representation valuable (46u5), the link from city construction to later interaction practical (s14i), and the metric structure recovery solid (xVxZ, 46u5). Below we clarify our contribution and add experiments that address the reviewers' questions:

  • A. A new setting, and a pipeline built on task-specific priors (s14i, 46u5). CityCode targets a new setting: a metric, parametric city that agents can operate on over long horizons. Air2Code builds it from sparse aerial observations with models adapted with structural priors and new mechanisms (e.g., ground-anchored scale calibration, structure-guided completion, constraint-checked assembly), rather than simply chaining existing models.
  • B. The results depend on our mechanisms and our models adapted with structural priors, not on closed-source models such as GPT and Gemini (xVxZ, s14i, 46u5). With open-weight models replacing either closed-source one, geometry stays comparable or better (building prism IoU 0.775 vs. 0.737; DSM RMSE 14.39 vs. 14.37 m), with a moderate drop in appearance that still leads all baselines. Removing our adapted models or mechanisms instead degrades results sharply: prism IoU 0.737 → 0.662 with the official TRELLIS.2 instead of our structure-aware generator, PSNR 14.46 → 13.44 dB without structure-guided appearance completion, and camera-depth RMSE 6.27 → 69.58 m without depth adaptation and calibration.
  • C. Data split (xVxZ). Training and test use two separate maps with no overlap, as in OpenFly.
  • D. Long-horizon tasks (s14i). Over 50 chains of 50 steps, the city state stays correct through edits and restorations. We add a road-graph evaluation (ground APLS 0.886) and compare long-horizon state maintenance and agent cost with the baselines on the same tasks.
  • E. Cost (xVxZ, 46u5). USD 1.00 and 32 min per 400 m region; no manual editing.

Questions raised by a single reviewer are answered with new experiments in the individual responses: additional VLM judges and building styles outside the training library (xVxZ), camera pose and intrinsic errors (46u5), and how reconstruction errors reach query answers (s14i). The revised paper and supplementary material are uploaded, with all changes, including the figure, table and writing fixes, marked in blue (summary in Part F).

"Sec." refers to the main paper, "App." and S-numbered tables/figures to the supplementary material, and "Part" to this General Response.

AuthorsOpenReview ↗

General Response (2/5)

A. Setting and contribution (s14i Q1, W1, Q3; 46u5 W1, W2)

Where the contribution lies (s14i W1; 46u5 W2). Not in a single component, but in three connected parts, now listed as the contributions in Sec. 1:

  1. New setting and representation (CityCode).

    • Input → output. From one low-altitude aerial RGB image per region with known cameras, CityCode constructs a fully metric, parametric 3D city: building instances, road geometry and connectivity with street blocks, and entity-linked 3D assets, kept as a versioned state that agents can operate on.
    • What is new. To our knowledge, it is the first city-scale parametric representation with recovered metric building instances and urban structure, rather than supplied as layouts or maps or generated from text (Table A1).
  2. A pipeline built on task-specific priors (Air2Code). A single image has no metric scale and leaves most facades unseen, and our ablations show that simply combining existing models does not solve this (46u5 W2; Part B2). Air2Code therefore combines:

    • Model adaptation with structural priors. The building generator, adapted from TRELLIS.2, is trained with footprint and height-to-footprint-ratio losses, and its sampled structure is corrected to the recovered building height; CityCode-Depth adapts MoGe2-Aerial to low-altitude views with metric-scale supervision.
    • Structure-and-observation-guided generation. Structure-guided appearance completion and the structure-aware generator keep generated buildings consistent with the recovered massing, location and scale.
    • Ground-anchored scale calibration and constraint-checked assembly. Depth is scaled against the ground, and an asset is bound only after ground-contact, road-intrusion and clearance checks pass.

    Replacing the closed-source models with open-weight ones preserves these results (B1).

  3. Evaluation and long-horizon use. CityCode leads on 21 of 22 metric-geometry and visual-quality metrics on CS-Aerial and UrbanScene3D (Tables 1–2). As a metric, parametric and versioned state, it lets agents query, edit, plan and restore earlier states over 50 chains of 50 steps; to our knowledge, this is the first exploration of such long-horizon tasks in an observed city (Part D).

Closest work (s14i Q1). Table A1. Only CityCode recovers metric building instances and urban structure from observations and keeps them as an operable, versioned city. ✓ = recovered from the input observations; – (given) = supplied as input; – (synthesized) = generated from text, not tied to a target city; – = not available.

Method (line of work)Input3D outputMetric geometry from observationsBuilding instances from observationsRoad graph + blocks from observationsEditsVersioned state + restoration
MLS-BRN, SingleRecon, DG-BRF (monocular building reconstruction)one off-nadir satellite image tileLoD1 boxes only✓ (heights only)✓ (LoD1 only)–––
Sat3DGen, Sat2Density++ (image-conditioned scene generation)one satellite image tilelocal scene✓ (surface only)––––
GaussianCity, CityDreamer4D (layout-conditioned city generation)semantic + height layoutcity– (given)– (given)– (given)––
MajutsuCity (design-driven)textcity–– (synthesized)– (road surfaces only)✓–
UrbanWorld2.0 (map-driven)OSM map + street-level RGBcity– (given)– (given)– (given)object edits–
CityCodeone aerial RGB per region + camerascity✓✓✓✓✓
  • City generation (GaussianCity, CityDreamer4D, MajutsuCity, UrbanWorld2.0) works at city scale, but takes its structure from layouts or maps or generates it from text, rather than recovering it from observations of the target city.
  • Editable scene representations (s14i Q1) maintain editable state in other settings: CityGML 3.0 for urban data exchange, 3D scene graphs (Hydra, ConceptGraphs) in robot mapping, and SceneCode in indoor generation. In city generation, however, validating edits, updating dependent entities and restoring earlier task states over long task sequences remain largely unexplored.

Revisions. Sec. 1 adds the contribution list (s14i Q1; 46u5 W1) and clearer transitions from representation to construction, generation and state updates (s14i Q3); Sec. 2 adds monocular building reconstruction to related work, and Table A1 appears as Table S22 in App. J.

AuthorsOpenReview ↗

General Response (3/5)

B. Do the state-of-the-art results come from leading closed-source models (GPT-image-2, Gemini 3 Pro Image) or from our pipeline, and how do methods compare given the same input? (46u5 W2–W5; xVxZ Q3; s14i Q2)

We answer with three experiments: replacing the closed-source models (B1; xVxZ, 46u5), ablating our adapted models and mechanisms (B2; 46u5, s14i) and comparing methods given the same input (B3; s14i, 46u5).

B1. Replacing the closed-source models with open-weight ones: each model has its strengths, and CityCode stays ahead of the baselines (xVxZ Q3; 46u5 W3, W5). xVxZ asks how much Table 1 and Table 4(b) would degrade with open-source alternatives, and 46u5 asks whether the gains come from these models. We replace the two closed-source stages of Air2Code with open-weight models, keeping inputs, seeds and all downstream code fixed (App. G.8, Tables S15–S16):

  • Structure extraction (building instance and semantic masks): Gemini 3 Pro Image → SAM 3.1.
  • Building completion: GPT-image-2 → FLUX.2-klein-9B.

The 3D building generator is already open-weight: it adapts TRELLIS.2 with structure-aware building priors on a building dataset (compared with the official TRELLIS.2 in B2).

(a) Structure extraction: Gemini 3 Pro Image → SAM 3.1. Full test city (79 regions):

MetricGemini 3 Pro Image (paper)SAM 3.1
Building instances in the city, [email protected] ↑79.0 %66.2 %
Road masks, IoU ↑0.7170.623
Building masks per view, instance F1 ↑0.5140.609
388 buildings matched by both: footprint IoU ↑ / XY error (m) ↓ / height MAE (m) ↓0.796 / 2.29 / 3.790.797 / 2.07 / 3.61
DSM RMSE (m) ↓14.3714.39

The open-weight SAM 3.1 gives comparable results: Gemini is stronger on city-level building instances and roads, SAM 3.1 on per-view masks and the geometry of matched buildings, and DSM RMSE is unchanged.

(b) Building completion: GPT-image-2 → FLUX.2-klein-9B. Single buildings: all 547 test buildings, as in Table 4(b). City level: one 480 m downtown region (77 buildings) re-rendered from 50 Table 1 cameras, with the rest of the scene unchanged.

AspectMetricGPT-image-2 (paper)FLUX.2-klein-9B
Building shapefootprint / prism IoU ↑0.842 / 0.7370.875 / 0.775
height relative error ↓0.1530.141
Appearance, single buildingPSNR ↑ / SSIM ↑14.46 / 0.77112.65 / 0.769
DINOv2 ↑ / LPIPS ↓0.57 / 0.5460.43 / 0.546
Appearance, city levelPSNR ↑ / SSIM ↑13.98 / 0.34613.67 / 0.328
LPIPS ↓ / DINOv2 ↑0.554 / 0.6680.555 / 0.657
VLM head-to-head (Table 1 judge), visual / GT-cond. ↑0.32 / 0.330.68 / 0.67
CostcompletionUSD 0.053 per building (API)5.9 s per building, single GPU

FLUX.2-klein also gives comparable results: it is better on building shape and is preferred by the VLM judge, while GPT-image-2 reproduces the observed facades more faithfully. City-level values use a different renderer and region from Table 1, so only the two columns are comparable; the VLM judge (DeepSeek V4.1 Flash), prompts and protocol are those of Table 1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.