- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper presents MobiSim, a generative AI-driven mobile network digital twin platform that integrates 3D urban environment generation, hybrid ray-tracing-based electromagnetic simulation, 5G RAN protocol emulation, and generative user behavior modeling within a unified framework. Its main contribution is a deployable, geography-aware simulator that can automatically synthesize city-scale network environments and expose standardized interfaces for network planning and optimization using conventional solvers, reinforcement learning, and LLM-based methods. The authors validate individual components using 3D scene comparisons, real-world received-power measurements, user mobility and traffic datasets, and several network optimization tasks. They additionally release synthetic radio-map, full-stack network, and 3D city datasets generated by the platform for downstream research.
Critical strengths
A substantive strength is the system-level integration of generative 3D environment synthesis, electromagnetic simulation, user behavior modeling, and 5G RAN operations within a single executable platform. This integration is technically nontrivial and could provide a convenient environment for studying cross-layer network optimization that is difficult to evaluate with isolated channel or protocol simulators. The paper also makes a meaningful effort to validate several components beyond purely synthetic examples, including comparisons against real received-power measurements across multiple datasets and frequencies, as well as real mobility and traffic datasets. These validations improve the practical relevance of the platform, even though they do not yet establish end-to-end digital-twin fidelity
Critical weaknesses
-
MobiSim primarily integrates existing 3D generation, ray tracing, and network simulation techniques. Its fundamental novelty over established platforms such as Sionna RT/PHY/SYS and recent generative wireless modeling approaches remains unclear. Moreover, the absence of physical–virtual synchronization makes its characterization as a digital twin questionable. These issues substantially weaken the significance and originality of the central contribution.
-
The evaluations primarily validate individual components rather than the complete simulation pipeline. Real-world electromagnetic validation relies on normalized correlations from single-BS measurements, without absolute prediction errors or multi-cell interference validation. Furthermore, no real-network experiments verify whether simulated optimization gains translate into actual performance improvements. Consequently, the evidence is insufficient to support the central fidelity claims or establish the practical reliability of the generated datasets and optimization results.
Questions for authors
- The motivation in the Introduction is substantially broader than what the proposed system actually addresses. For example, the paper motivates MobiSim by the increasing complexity of the electromagnetic environment caused by higher spectrum usage, yet MobiSim only performs forward simulation from predefined network configurations. It does not perform spectrum sensing, wideband electromagnetic monitoring, measurement-driven reconstruction, or spatial extrapolation of the actual radio environment. More generally, several challenges introduced at the beginning of the paper are not connected to corresponding technical solutions or evaluations.
- The near-field claim in Section 3 is particularly problematic. The authors explicitly identify both near-field and far-field propagation as a fundamental challenge, yet the proposed electromagnetic layer is based on conventional geometric ray tracing, standard antenna patterns, Jones-matrix interactions, and distance-dependent free-space spreading. No spherical-wave channel model, near-field array response, reactive-field modeling, or full-wave treatment is introduced or evaluated. Therefore, the presented method does not substantiate the claimed near-field modeling capability, suggesting that the wireless-domain framing of this challenge is insufficiently precise.
- The central novelty relative to existing wireless simulation platforms and recent generative wireless-environment models is not convincingly established. Sionna already provides RT, PHY, and SYS modules, with multi-cell system-level simulation, link adaptation, power control, and user scheduling, and its official workflow directly couples Sionna RT with Sionna SYS. Meanwhile, generative approaches such as RadioDiff and WiFi-Diffusion have already formulated radio-environment construction as generative modeling problems. The paper does not sufficiently clarify what methodological capability is fundamentally new beyond integrating several existing components into one pipeline.
- I am also not convinced that MobiSim should be characterized as a mobile network digital twin rather than a synthetic network simulator. ITU-T Y.3090 defines a digital twin network around real-time interactive mapping between the physical network and its virtual representation. ITU MobiSim does not appear to ingest online measurements from an operational network, synchronize its states with a physical network, or maintain a physical-to-virtual feedback loop. Its pipeline instead synthesizes a virtual network from map data and configured parameters, which is conceptually closer to a geography-conditioned simulator.
- Several other key motivations in the Introduction are similarly not covered by the proposed methodology or experiments. The paper refers to spectrum allocation, antenna beamforming, and increasingly complex electromagnetic environments, but the demonstrated tasks are mainly BS placement, sector azimuth/downtilt adjustment, and TDD/PRB scheduling. The gap between the broad problem framing and the actual supported capabilities makes several claims appear overstated.
- The layer-wise validation does not establish the claimed full-stack end-to-end fidelity. The 3D generation, electromagnetic layer, user model, and network optimization are evaluated separately, often using different cities and datasets. There is no real-world experiment validating the complete chain from scene reconstruction through propagation and RAN operation to user-level KPI outputs. Consequently, accurate performance of individual components does not establish that errors do not accumulate across the full pipeline.
- The 3D-generation evaluation is also difficult to interpret as evidence of electromagnetic fidelity. MobiSim uses additional street-view images while several baselines use only OSM inputs, which the paper itself acknowledges prevents isolating the contribution of the generation method. More importantly, LPIPS, MS-SSIM, boundary alignment, and aesthetic scores evaluate rendered visual similarity rather than building height, surface geometry, or material accuracy. The authors explicitly acknowledge that these metrics do not establish 3D geometric or material fidelity. For ray tracing, visual similarity is not sufficient evidence of electromagnetic equivalence.
- The real-world electromagnetic validation is too weak to support the repeated “high-fidelity” claims. The main results compare normalized and smoothed received-power curves and report Pearson and Spearman correlations, but do not provide absolute received-power MAE/RMSE, systematic bias, coverage-threshold accuracy, or calibration error. A simulator can achieve high correlation while having a large absolute power error, which is critical for coverage, handover, and scheduling decisions. Moreover, the real-world validation uses single-BS measurements and therefore does not validate the claimed multi-cell interference and SINR accuracy.
- The claimed unbiasedness of the Monte Carlo estimator requires stronger theoretical justification. The interaction proposal assigns zero geometric reachability to outgoing directions that do not directly project onto the receiver grid, which appears capable of assigning zero sampling probability to paths that could reach the receiver after subsequent reflections or diffractions. If a physically valid path has nonzero contribution but zero proposal probability, importance weighting cannot recover it. The authors should either prove full support over all contributing paths or introduce an exploration component that guarantees nonzero sampling probability.
- The optimization experiments demonstrate that the simulator can be optimized internally, but they do not establish counterfactual fidelity with respect to a real network. Improving SINR or throughput inside MobiSim only shows that an optimizer can exploit the simulator's objective landscape. A substantially stronger digital-twin validation would require showing that a configuration change predicted to improve performance in MobiSim produces a comparable change in measurements from the corresponding physical network.
- The scalability claims also require clarification. The paper reports approximately 5–8 seconds for a warmed-up electromagnetic computation, while the PPO experiments contain roughly training steps. It is unclear how many of these training steps invoke the full electromagnetic simulator, whether radio maps are cached or approximated, and what the total GPU-hour cost is. Without this information, the practical scalability of MobiSim as an RL environment is difficult to assess.
- Finally, the scientific value of the released dataset is insufficiently demonstrated. The radio maps, channel-quality metrics, traffic traces, scheduling records, and RAN events are predominantly synthetic outputs of MobiSim rather than measurements from operational networks. Large synthetic wireless datasets generated by established ray-tracing and system-level simulation tools already exist, including datasets based on mature and extensively validated propagation engines. The manuscript does not show that MobiSim-generated data are more accurate, enable a fundamentally new learning problem, or yield better transfer to real measurements. Since the validity of the dataset is entirely inherited from the simulator itself, releasing a larger amount of simulated data does not provide independent evidence of either dataset quality or simulator fidelity.
- Relatedly, the network-layer fidelity is largely asserted rather than validated. Although the paper implements a number of 3GPP-inspired procedures, the network-layer evaluation mainly demonstrates qualitatively plausible daily traffic patterns rather than comparisons with real handover, PRB allocation, MCS, throughput, or RAN-event traces. This makes the “full-stack” and “complete 5G RAN” claims significantly stronger than the supporting empirical evidence.