acceptodds
Under review as a conference paper at ICLR 2027

CityDeploy-Bench: Learning Collective Utility for Physics-Grounded Spatial Planning

Abstract

Automating city-scale wireless deployment remains challenging under complex urban propagation and network-wide interference. We introduce CityDeploy-Bench, a benchmark that reframes multi-transmitter deployment as physics-grounded spatial set planning under a unified ray-tracing verifier. The benchmark separates utility representation from planning dynamics, enabling controlled comparison between direct scalar rewards, relational models, and higher-order interaction structures across diverse planners. Our experiments reveal a clear transition in planning behavior as physical coupling grows. Deployment quality becomes increasingly dependent on whether the learned utility captures collective transmitter interactions, whereas stronger search alone cannot compensate for missing relational structure. This establishes multi-transmitter deployment as a coordination problem over physically interacting sets rather than a collection of independent spatial decisions. We release CityDeploy-Data and the benchmark framework as a reproducible testbed for research linking decision learning with physically grounded wireless network design.

Primary area:
datasets and benchmarks
Keywords:
spatial planning, decision making, relational learning, energy-based models, diffusion models, graph neural networks, physics-grounded learning, combinatorial optimization, wireless networks, benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NiGNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper provides a benchmark that reframes multi-transmitter deployment as physics-grounded spatial set planning under a unified ray-tracing verifier. On the top, the paper shows the possibility to compare some planning methods. Overall, the paper is easy to follow. But significant over-simplification exists.

Critical strengths

The paper put some efforts in simulating ray tracing for benchmarking the multi-cell deployment.

Critical weaknesses
  1. The benchmark is simulated under over-simplified assumptions, including the material priors (only three, very coarse classification, whereas more material are general involved), the configuration of TX and RX single-element isotropic antenna with vertical polarization and zero azimuth orientation (whereas MIMO is standard in current cellular networks and coordinated multi-point in a building is common), and single carrier frequency (where inter-cell interference can be largely mitigated by different frequencies, and that is what makes cellular networks be called cellular networks).

  2. I doubt the practical applicability and/or the target scenario of the benchmark. Cellular deployment is a complicated problem, which depends on the technical networking optimization and the practical deployable factors, such as agreement from residents. So, the current deployable mask is over-simplified as well and might not have a easy solution.

Questions for authors

Address my concern over assumptions and the target application scenario.

3 comments
AuthorsOpenReview ↗

Response to NiGN (1/2): W1 — Radio assumptions

W1: Radio assumptions

Thank you for recognizing the effort behind the benchmark and for questioning its modeling scope. CityDeploy-Bench studies how to learn collective deployment utility and search for effective transmitter sets under specified physical conditions. Shared scenes, utility labels and ray-tracing evaluation make this learning/planning problem reproducible. We have tested several of the disputed assumptions directly.

The reference profile uses 3.65 GHz, 20 MHz, 24 dBm TX power and 10 m/1.5 m TX/RX heights. It retains urban blockage, specular/diffuse reflection, refraction and diffraction. Fixed antenna and resource settings isolate spatial planning; they do not constitute a complete cellular-system model.

Frequency reuse. We agree that frequency separation can mitigate interference. Reuse-one operation is nevertheless a legitimate cellular regime: 3GPP TR 36.902, §4.9 explicitly discusses co-channel interference and PRB-level coordination. An operational 4G/5G measurement study also reports persistent inter-cell interference (Ding et al., 2025). Thus, the reference investigates a relevant interference-limited condition, while frequency coordination is an important extension.

We reevaluate 90 fixed layouts: two utility models × three scene/TX tasks × three training seeds × five planning seeds. A geometry-based channel assignment partitions the same 20 MHz/51 PRBs into K groups. Noise, per-resource-element received power and throughput are recomputed using each group's bandwidth; only co-channel TXs interfere. Table 1 holds power spectral density fixed, so per-TX total power decreases with allocated bandwidth.

Table 1. Fixed-layout joint coverage JM (%) under frequency reuse.

Reuse factor KH4CI
160.33±1.6462.11±3.36
270.17±2.1170.30±3.34
370.88±1.3471.52±3.70

JM is the fraction of road cells meeting all four thresholds: path loss, SS-RSRP proxy, SINR and effective throughput. H4=HPEM with orders 1–4; CI=IN with common input features. We average planning repeats within each task/training seed, then equally weight tasks; mean±sample SD is across three training seeds. The tasks are London/256 m/4 TX, New York/512 m/7 TX and Tokyo/512 m/9 TX.

Reuse 1→3 improves JM by 10.55/9.41 percentage points for H4/CI, even after accounting for reduced bandwidth and TX power. With fixed total TX power, the corresponding K=3 values are 72.06±1.29/72.70±3.61%. This directly quantifies the reviewer's proposed mitigation without granting extra spectrum. It also shows why conclusions must identify their resource regime. This test assumes frequency-flat gains over the original 20 MHz; it is not a full frequency-selective scheduling study.

Antennas and coordination. For the isotropic reference, zero azimuth does not impose a directional gain preference. Single-element vertical polarization provides a controlled starting point. To test antenna-pattern shift, we use a TR 38.901 directional TX element with 8 dBi peak gain, reduce conducted power to 16 dBm to retain 24 dBm peak EIRP, and point horizontal boresights toward the scene center; RXs remain isotropic. We rerun planning and fine-tune the utility models with 5,184 training labels, using 528 validation labels for checkpoint selection and 528 separate test labels.

Table 2. Directional-profile Diffusion planning, JM (%). Statistics follow Table 1; each scene group contains three tasks. Held-out scenes are within training cities, not entirely unseen cities.

ScenesModelBefore adaptationAfter adaptation
SeenH453.39±1.2352.94±0.12
SeenCI54.29±2.4854.74±1.46
Held-outH460.94±1.4361.99±1.02
Held-outCI60.57±1.0961.63±2.42

The benchmark supports both evaluation under the changed profile and learning with its new labels. Adaptation effects are modest and mixed, so we do not claim configuration-invariant performance. This experiment tests directional elements, not full MIMO, beam optimization or CoMP. Those require explicit array, precoding and coordination policies, and we will state this boundary clearly. The study's value is that these physical assumptions can be varied and their consequences measured within a common planning/evaluation pipeline.

AuthorsOpenReview ↗

Response to NiGN (2/2): W1, W2, Q — Materials and siting

W1 (materials), W2 and Q: Fidelity and application scope

Material priors. The three priors use Sionna's ITU-based concrete, medium-dry-ground and wet-ground models, with frequency-dependent electromagnetic coefficients (documentation). They are physical priors rather than measured material maps. The semantic assignment remains coarse: buildings/roads use concrete, land/green areas use medium-dry ground, and water uses a wet-ground proxy. Standards-derived coefficients do not establish that these assignments match each actual surface.

We therefore performed paired reevaluation of 120 fixed layouts per perturbation. Multiplying material conductivities by 0.8/1.2 yields mean absolute JM changes of 0.52/0.42 percentage points. These tests assess local coefficient sensitivity only; they do not validate the three-class assignment or rule out larger errors from glass/metal facades, vegetation or moisture. Site-specific accuracy requires richer material information and measurement calibration. We will explicitly distinguish simulation-grounded verification from empirical validation.

Target application. The concrete task is outdoor coverage-oriented placement of a fixed number of TXs in an urban region, given a candidate-site set and a radio/resource profile. It models the technical planning stage: choosing a jointly effective set from admissible locations. Resident consent, ownership and installation permissions determine admissibility through external processes. The benchmark does not infer those permissions from a map or certify that every road position is installable.

Once authorized sites are supplied, the planning constraint can be written as

P⊆Cgeometry∩Cauthorized,∣P∣=N.P\subseteq\mathcal C_{\mathrm{geometry}}\cap\mathcal C_{\mathrm{authorized}},\quad |P|=N.

Crucially, candidate locations and the area requiring service are separate inputs. Restricting where TXs may be installed must not shrink the area on which coverage is evaluated. This lets the benchmark test the technical consequences of site availability while keeping the coverage objective fixed.

We tested this distinction by replanning under restricted candidate sets. We retain 50% or 25% of candidate-bearing 20 m blocks, with three mask seeds, and separately restrict candidates to a 2 m band inside the road-mask boundary. The latter is a roadside geometry proxy, not a map of surveyed poles or approved installations. The reward models remain frozen; restrictions enter the search constraints and final feasible projection. TX counts, 20 m minimum separation and the original road evaluation area remain unchanged.

Table 3. Restricted-site Diffusion planning, JM (%). H4=orders 1–4 HPEM; CI=common-input IN. Both use 12,020 search reward evaluations plus 20 final reranking evaluations.

Allowed candidatesH4CI
Original road candidates64.60±1.3366.10±2.13
50% of candidate blocks retained63.90±1.2165.33±1.17
25% of candidate blocks retained63.48±0.3263.99±1.56
Within 2 m of road-mask boundary61.95±1.3859.43±2.67

These use the same three London/New York/Tokyo tasks as Table 1, with three training seeds and five planning seeds. We average planning/mask repetitions within task/training seed, equally weight tasks, and report mean±sample SD across training seeds. Table 3 uses new searches with a common candidate archive; its original-road row is not the fixed-layout baseline of Table 1.

Restricting H4 to the boundary band changes JM from 64.60% to 61.95%; retaining 25% of candidate blocks gives 63.48%. The results demonstrate planning with substantially reduced geometric availability, while quantifying the resulting coverage. They do not validate real permissions or guarantee feasibility for arbitrary candidate sets.

Research value. Real deployment requires several coupled decisions; a reproducible benchmark for spatial selection is useful within that larger workflow. Operator-ready validation additionally requires calibrated propagation, operational radio policies and verified site permissions; we will make this scope explicit. The ML challenge addressed here is learning and optimizing the utility of an entire spatial set under a costly physical evaluator and changing feasible regions. CityDeploy-Bench provides open scenes, deployment labels and a common evaluator for comparable studies of this challenge. The added experiments make spectrum allocation, antenna patterns and site availability testable conditions, allowing researchers to examine where learned planning succeeds and where it fails.

Reviewer_NiGNOpenReview ↗

Feedback to the responses

Thanks for the input. Although the authors put a lot of effort into demonstrating the workflow's capabilities, their responses, in my opinion, also implicitly reveal the complexity and complications of the investigated scenarios, which the paper fails to address sufficiently. Moreover, their final statement suggests the title is somewhat overstated, or at least misleading. Therefore, I choose to maintain my score.

Reviewer_p2gmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies wireless network planning using machine learning techniques. It introduces CityDeploy-Bench, a framework for multi-transmitter (TX) wireless deployment. Experiments evaluate joint-metric (JM) coverage, planning efficiency, target-coverage success, stability, and geographic adaptation. The results show that adding transmitters can reduce joint service coverage due to interference, while higher-order relational representations can improve planning performance.

Critical strengths
  1. This is an interesting application of machine learning and AI to telecommunications. The problem is relevant, and the evaluation provides meaningful insights into the non-monotonic nature of wireless deployment. In particular, it shows that adding a new transmitter could actually reduce JM coverage, demonstrating the importance of accounting for interference and interactions among transmitters rather than optimizing individual locations independently. These results are not, however, surprising, as this non-monotonic nature of wireless deployment is known to the wireless community. The interesting part is quantifying the severity of this effect.

  2. The separation of utility representation from planning dynamics through a common interface and physical verifier is an interesting contribution. This enables systematic comparisons across different learning architectures and optimization methods and provides a useful foundation for subsequent research.

Critical weaknesses
  1. The paper presents an interesting application of AI techniques in other fields, but it does not necessarily present a new AI methodology or push the boundaries of AI. In this sense, its methodological contribution to AI is limited. I think this paper may be appreciated more in domain-specific journals and conferences in telecommunications, such as IEEE INFOCOM or IEEE Transactions on Wireless Communications.

  2. HPEM differs from the baseline representations in model capacity, positional features, training objectives, and sample weighting. Hence, the reported improvements cannot be attributed exclusively to higher-order interactions, as these other differences could also explain the results.

  3. Experiments use small simulated urban regions, simplified radio configurations, and geographically separated scenes from cities already included in training. There is no validation against real measurements or evaluation on an entirely unseen city.

Questions for authors
  1. Can the authors provide comparisons between techniques using the same model capacity, input features, training objectives, sampling distributions, and inference budgets? This would help to isolate the contribution of higher-order relational structure.

  2. What evidence supports the claim that stronger search cannot compensate for missing relational structure? Note that in Table 23, some of the larger-budget configurations outperform smaller-budget fourth-order configurations.

  3. How well does the proposed solution generalize to new settings (e.g. unseen cities, larger deployment regions, and more realistic radio configurations)?

  4. Why do the authors not consider reinforcement learning techniques as planning baselines for this problem? Would these not provide a useful comparison, given that transmitter deployment could be formulated as a Markov Decision Process?

5 comments
AuthorsOpenReview ↗

Response to p2gm (1/4): W1, W2, Q1

W1,W2,Q1

Our response has four parts: (1) contribution and controls; (2) search budgets and utility error; (3) transfer and radio settings; (4) reinforcement learning.

Thank you for recognizing the common utility interface and physical verifier. We added controlled comparisons, whole-city transfer tests, and RL baselines.

CityDeploy-Bench studies how to learn an interacting spatial set's utility so that optimizing predictions produces physically effective decisions. It provides simulator-verifiable data, independently replaceable utility models/planners, and evaluation of prediction, ranking, deployment quality, cost, and transfer. HPEM supplies a scene-conditioned, permutation-invariant utility family with configurable explicit interaction order.

The contribution is an experimental framework and evidence about learning to plan with interacting sets: when prediction improvements translate into better decisions, when search exploits prediction error, and when amortized policies are useful. This datasets-and-benchmarks contribution does not require one architecture to win every setting.

The original model×planner comparison evaluates complete systems. To test architectural attribution, we now control:

  • Data: identical splits and 38,162 unique training deployments, no family duplication, common scene/cardinality batching.
  • Inputs: common encoder architecture and coordinate, size, cardinality, Fourier/local-field feature types. CI also receives pair geometry/centroid fields; encoder weights are independently learned.
  • Training: MSE, AdamW, learning rate 1e-4, batch 32, up to 100 epochs, patience 15, validation-MSE selection. Composite loss is a separate ablation.
  • Capacity: H2w and CI differ from H4 by only 0.97% and 0.64% in trainable parameters.

Table 1. Controlled prediction and Diffusion planning. Hk=HPEM orders 1…k; H2w=widened H2; CI=common-input IN. JM is Sionna-verified joint coverage (%); MAE uses percentage points (pp). Tasks: London-07/4 TX, New-York-04/7 TX, Tokyo-03/9 TX.

ModelParams(M)MAE↓JM@12,020↑JM@48,020↑
H10.33115.27±0.2759.18±2.0157.41±1.91
H20.5987.44±0.0358.11±2.3360.71±2.33
H2w1.1437.52±0.0758.66±1.5862.34±2.80
H30.8657.64±0.1859.65±0.9259.63±2.54
H41.1327.37±0.1360.33±1.6458.49±2.64
CI1.1407.36±0.0862.11±3.3762.31±0.81
H4−F1.13014.75±0.1455.90±2.2154.21±0.92
H4+L1.2717.31±0.0560.50±3.0660.09±0.70

Unless stated otherwise, planning averages five seeds per training seed/task, then tasks equally, reporting mean±sample SD across three training seeds. MAE equally weights held-out scene/cardinality groups. H4−F jointly removes Fourier, subset-geometry and centroid-field features. H4+L adds ranking and auxiliary losses, each weighted 0.25; parameters include auxiliary heads.

The controls identify both benefits and boundaries. H2 is part of the HPEM family; pairwise/higher-order variants predict utility better than H1 under this protocol. Feature removal raises H4 MAE from 7.37 to 14.75 pp. However, H2w, H4 and CI remain close, so adding explicit third-/fourth-order terms is not uniformly beneficial. Matching feature types and parameter counts tightens control; order-specific subsets and FLOPs still differ.

Explicit edge order is not the full predictor's interaction order: IN uses nonlinear updates/readout after pairwise aggregation, and HPEM—including H1—has a nonlinear global head. These are comparisons of architectural inductive biases, not proofs that alternatives cannot represent collective effects. We retain the controls and treat order selection as a task-dependent design question, distinguishing it from the original complete-system comparison.

AuthorsOpenReview ↗

Response to p2gm (2/4): Q1, Q2

Q1,Q2

We compare Diffusion and PSO under matched reward-query and synchronized search-time budgets, paired seeds, common feasibility handling, and a shared queried-candidate archive with final projection/reranking. Proposal distributions remain algorithm-specific because they define each planner.

Table 2. Matched-budget JM (%). Models, tasks and statistics follow Table 1.

ModelPlanner12,020 queries5 s20 s
H2wDiffusion63.78±1.0463.91±0.5364.18±1.74
H2wPSO58.76±3.6261.01±1.1063.19±0.98
H4Diffusion64.60±1.3364.76±1.4362.93±1.03
H4PSO59.79±2.1961.37±2.5360.86±1.57
CIDiffusion66.10±2.1365.74±2.9265.97±0.60
CIPSO59.84±1.9362.20±2.6363.35±1.51

The query setting uses 12,020 search evaluations plus 20 reranking evaluations; two setup probes are separate. Search-time budgets exclude encoding, setup, projection/reranking and ray tracing. Mean search-plus-projection/reranking times are 5.26/20.26 s. Table 1 uses the earlier sampler-output rule; Table 2 uses a common candidate archive. They are separate experiments, not one budget curve.

Stronger search can compensate in particular settings: Table 1's H2w at 48,020 queries reaches 62.34%, versus H4's 60.33% at 12,020. We will replace the categorical claim. The key question is whether accurately optimizing learned utility guarantees a good physical decision.

To isolate this, we eliminate search error on finite problems: three scenes×three geometry-only candidate pools, enumerating all feasible 4-TX subsets of 24 sites (up to 10,626 per pool). Each model's exact predicted optimum is compared with the exact physical optimum over the same cache-defined feasible sets.

Table 3. Physical JM at the exact predicted optimum. Nine instances cover London-07, New-York-06 and London-08; instances are averaged within each training seed. Mean finite physical optimum: 92.55%.

ModelPhysical JM↑Gap(pp)↓
H159.31±0.7133.24±0.71
H277.86±1.4514.69±1.45
H2w74.26±2.2418.29±2.24
H477.27±0.5215.28±0.52
CI79.36±2.1513.20±2.15

A 13.20–33.24 pp physical gap remains despite zero optimization error for the predicted objective. This distinguishes inadequate optimization from incorrect ranking of physical decisions. Separately, verifying all 20 retained candidates from 45 searches finds an average 11.62 pp gap between the predicted-best and physically best candidate in the same set (training seed 42; local ranking regret).

The cache was checked against 72 direct multi-TX simulations: mean/max discrepancies 0.37/2.10 pp. Exactness applies to this finite cached problem, not the continuous domain. This does not establish fourth-order necessity or a universal physical-performance ceiling: inexact search can select a physically better, lower-predicted layout.

Physical planning quality depends jointly on utility fidelity and search; increasing predicted utility alone does not guarantee better physical deployment. The benchmark makes this distinction testable, supporting research on decision-aware training, ranking calibration and selective physical verification.

AuthorsOpenReview ↗

Response to p2gm (3/4): W3, Q3

W3,Q3

We evaluate geographic, scale and radio shifts separately because they test different capabilities. Coverage is JM (%); statistics follow Table 1.

Unseen cities. In strict London/Tokyo holdouts, all ten target-city scenes are excluded from training and validation. Four architectures are retrained from scratch with three seeds; target-city labels never select checkpoints. Diffusion uses 12,020 search queries and five planning seeds on 128 m/2 TX, 256 m/4 TX and 512 m/7 TX tasks.

Table 4. Whole-city transfer. Reward/IN/NRI are native architectures with common MSE training; this is geographic transfer, not capacity-controlled attribution.

ModelLondonTokyo
Reward62.62±3.2758.54±1.52
IN68.86±0.8465.54±0.11
NRI69.25±2.7066.61±0.47
H471.34±4.0367.86±1.89

H4 has the highest macro mean in both folds, exceeding Reward by 8.72/9.32 pp in London/Tokyo. This directly extends evaluation beyond unseen scenes within training cities. It does not establish superiority on every task or isolate fourth-order effects.

Larger regions. Existing models are evaluated at 0.8075/4.25 km², retaining 1 m radio-map cells.

Table 5. Scale/density stress tests with Diffusion. CI=common-input IN.

RegionSize(m)TXsH4CI
london_11950×850738.68±1.2438.30±1.86
london_11950×850944.64±1.1044.52±2.36
shanghai_112,500×1,700921.72±1.6322.49±0.79

The pipeline runs beyond the original 512 m regions; low coverage quantifies the difficulty of sparse infrastructure over larger areas. These regions partly overlap source areas and use at most nine TXs, so this tests scale/density, not unseen geography or larger TX cardinalities.

Radio configuration. The reference uses 3.65 GHz, 20 MHz, 30 kHz spacing/51 PRBs, 24 dBm conducted power, 10 m TX/1.5 m RX heights and explicit resource sharing. Sionna RT includes LOS, specular/diffuse reflection, refraction and diffraction. Standards-derived settings provide a reproducible physical task but do not imply measurement calibration.

We test a TR 38.901 directional element, reducing conducted power by 8 dB to preserve peak EIRP. Adaptation uses 5,184 training labels; 528 validation labels select checkpoints, with 528 separate test labels.

Table 6. Directional Diffusion before/after RF adaptation. Held-out scenes belong to training cities, unlike Table 4.

ScenesModelBeforeAdapted
SeenH453.39±1.2352.94±0.12
SeenCI54.29±2.4854.74±1.46
Held-outH460.94±1.4361.99±1.02
Held-outCI60.57±1.0961.63±2.42

Held-out means improve by approximately 1.05/1.07 pp for H4/CI, with mixed effects on seen scenes and other planners. Further tests cover frequency reuse, ±20% conductivity and candidate-site restrictions. At fixed power spectral density, with throughput recalculated for reduced bandwidth, reuse 1→3 changes H4 fixed-plan coverage from 60.33% to 70.88%. Restricting sites to a 2 m road-boundary band yields 61.95±1.38% for H4/Diffusion, with evaluated road area unchanged. This is a geometric installation proxy.

We lack real-measurement validation and do not model full MIMO/beam optimization, traffic-adaptive scheduling or site ownership. For 15 layouts, each averaged over three ray seeds, increasing rays/TX from 5M to 10M changes coverage by 1.06 pp on average (maximum 3.31 pp), so small differences merit caution. These studies expand the benchmark's tested operating conditions and quantify its boundaries, supporting reproducible research on transfer and physical sensitivity.

Reviewer_T1SCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CityDeploy-bench, which is a benchmark for city-scale wireless multi-transmitter deployment as physics-grounded spatial set planning. It also provides data of the multi-transmitter deployments and findings of physical coupling of the transmitters with simulator verified evaluation of different service and infrastructure related aspects. The paper is interesting and well written, but its contributions are not at the level of the ICLR publications.

Critical strengths

The benchmark is very valuable on its research field (wireless communications). Its development and evaluation via experimenting are well discussed and the opened dataset is very beneficial for the future research and development.

Critical weaknesses

The relevant state-of-the-art papers are given in the Introduction, but there is no clear gap statement, what the prior work fails to provide and what is the need that CityDeploy-Bench fills. There is also very limited connection to the related modeling literature but only some menitons on previous work on sets and graph methods. Also the evaluation protocol differences are not well specified although individual aspect are discussed. As a summary, this leaves the novelty compared to previous benchmarks ambiguous. Additionally, I think that while the methodological contributions are interesting, they are not at the level of an ICLR paper, but more on valuable engineering aspects.

Questions for authors

What is the research gap your work addresses, ie the main novelty? How do you see the relevance of the contribution of your work on the ICLR community?

3 comments
AuthorsOpenReview ↗

Response to T1SC (1/2): Research gap and practical value

Research gap and relation to prior work

Thank you for recognizing the benchmark's value and the benefit of releasing the dataset. Your review identifies a central presentation issue: the introduction lists relevant methods without making the missing experimental capability explicit. We will state the gap directly:

The gap is a reusable evaluation framework that distinguishes the contributions of learned set-utility representations and search to physically verified deployment decisions. CityDeploy-Bench makes utility representation and planning separately replaceable, while holding the task, deployment constraints and physical evaluator fixed.

The distinction from representative existing resources is:

Resource/workPrimary objectEvaluation focus
DeepMIMOGeometry-dependent wireless channelsReusable channel data for learning communication algorithms
RadioMapSeer and extensionsPathloss/ToA radio mapsRadio-map estimation and localization
Reward-guided AP planningIndoor deployments with a unified learned rewardGenerative planning and reward-guided search
CityDeploy-BenchFeasible sets of simultaneously active TXs and their collective utilityCrossed utility-model × planner evaluation of physical decision quality, cost and transfer

These resources provide important foundations and can support deployment research. CityDeploy adds a common deployment-level task and experimental protocol: feasible action sets, joint service labels, interchangeable utility models/planners, geographic splits and independent simulator scoring. The contribution lies in making their interaction systematically testable. We do not claim that wireless planning, ray tracing or learned rewards are individually new.

This distinction matters because a promising individual site need not be useful after adding other transmitters: serving-cell reassignment, redundant coverage and co-channel interference change its contribution. We provide 54,193 unique deployments across 60 scenes in six cities, paired with dense physical responses, so models can learn and be evaluated on this collective utility across configurations.

Practical significance. The benchmark connects directly to planning decisions: where to place a fixed number of transmitters, how many are needed to reach a service target, and when densification improves coverage or increases interference. Its verified layouts and coverage maps can inform candidate-site screening, site-count–coverage trade-offs and diagnosis of underserved areas before detailed engineering design. This provides a reference for comparing design alternatives under explicit assumptions. Field deployment would additionally require measurement calibration, realistic traffic and operational/site constraints; the benchmark supports that engineering workflow through reproducible simulation evidence.

Connection to modeling literature. We will make the architectural relationships explicit:

  • Deep Sets motivates permutation-invariant pooling: TX ordering must not change predicted utility.
  • Interaction Networks and NRI motivate explicit pairwise messages and inferred relation types. Our adaptations predict static deployment utility; they are not reproductions of the original dynamics-learning tasks.
  • Hypergraph neural networks motivate modeling relations among multiple elements. HPEM provides scene-conditioned subset potentials with configurable explicit order, making this inductive bias an experimental variable. Its potentials are predictive components, not identified electromagnetic interaction energies.

This comparison concerns inductive bias and learnability, not a claim that pooled or pairwise networks cannot represent collective effects. Nonlinear aggregation/readout can express such effects, and higher explicit order need not always be preferable.

The work also connects to task-based learning: prediction quality should be judged by the decisions it induces. CityDeploy supplies a physically verifiable setting with nonconvex spatial constraints, interference-dependent utility and a costly evaluator. It enables research on decision-aware objectives, relational representations and planner-induced distribution shift. Our next response specifies the evaluation protocols and the evidence this framework reveals.

AuthorsOpenReview ↗

Response to T1SC (2/2): Evaluation protocols and community value

Evaluation protocols and relevance to ICLR

We agree that the protocol distinctions should be visible in one place. The benchmark evaluates three tasks with different questions:

TaskControlled conditionReported outcome
Fixed cardinalitySame scene, TX count, feasible domain and radio profileSimulator-verified joint coverage, marginal coverage, planning cost and seed variability
Target coverageSame service target and ascending TX-count scheduleSuccess rate, first successful tested TX count and cumulative cost
Transfer/adaptationDeclared geographic split and target-supervision allowanceZero-shot/few-shot/full-pool planning quality on disjoint evaluation deployments

The first successful TX count is not a proven global minimum. Scene holdout and whole-city holdout are distinguished; our supplementary London/Tokyo city-holdout tests exclude the entire target city from both training and checkpoint-selection validation.

We also distinguish system comparison from controlled attribution. The original four-model × nine-planner matrix compares complete systems through a shared interface. It does not by itself isolate architecture or equalize compute. Supplementary controls use identical unique training deployments and splits, common input-feature types and MSE training, with widened HPEM-2 and common-input IN within 1% of HPEM-4's parameter count. Order-specific computation still differs. Separate Diffusion/PSO experiments match reward-query or synchronized search-time budgets and use common candidate retention/projection/reranking. Search, reranking and physical verification costs are accounted for separately. These distinctions will be consolidated into a protocol table.

What scientific question does this enable? A planner can optimize predicted utility successfully yet select a poor physical deployment. We tested this directly by removing search error on nine finite instances: three scenes × three geometry-only pools of 24 candidate sites, enumerating every feasible 4-TX subset (up to 10,626 per pool). For each model we evaluate the exact predicted optimum against the exact physical optimum over the same cache-defined feasible set.

Utility modelPhysical joint coverage (%)↑Gap to finite physical optimum (pp)↓
HPEM-159.31±0.7133.24±0.71
HPEM-277.86±1.4514.69±1.45
HPEM-2, widened74.26±2.2418.29±2.24
HPEM-477.27±0.5215.28±0.52
IN, common inputs79.36±2.1513.20±2.15

The mean finite physical optimum is 92.55%. HPEM-k includes orders 1 through k. Values are mean±sample SD across three training seeds after averaging the nine instances per seed. The cache was checked against 72 direct multi-TX simulations: mean/max discrepancies were 0.37/2.10 percentage points. Exactness applies to these finite cached problems, not the continuous deployment domain.

All five models retain a physical decision gap even when their predicted objective is optimized exactly. This separates utility-ranking error from inadequate search. It does not mean that approximate search can never find a physically better layout; it shows why increasing predicted utility alone is insufficient. The results also make interaction-order selection an empirical question: fourth order is not uniformly best, and the common-input IN is competitive. These outcomes are informative findings of the benchmark rather than a requirement to establish one universal winner.

Value to the ICLR community. CityDeploy makes learning representations that remain useful under optimization a reproducible research problem. Its physical reference exposes how prediction errors become decision errors, enabling work on relational inductive biases, decision-aware training and allocation of verifier queries. Releasing aligned scenes, deployment labels and executable evaluation tools lowers the entry barrier: ML researchers can develop representations and planners without first reconstructing a wireless simulation/data pipeline. Shared tasks, splits and verification rules make subsequent methods comparable and their failures inspectable. The framework also lets wireless researchers test whether a gain comes from a utility model, search or a changed physical setting. This shared experimental foundation connects methodological progress to consequential infrastructure decisions and supports cumulative research across the two communities.

We will revise the introduction around this research gap, add the explicit literature/protocol comparison, and distinguish the original system-level findings from the supplementary controlled evidence. This will make the benchmark's scientific contribution, alongside its practical infrastructure, substantially clearer.

Thank you for your replies and efforts on improving the paper. While I think the exaplanations are convinsing, they don't unfortunately remove my concerns related to the novelty and relevance to this community and therefore I will keep my recommendation as is, but encourage the authors to resubmit the paper to a more suitable forum.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.