- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors present VIDA-GEO, a multi-agent pipeline that proposes local, policy-aware edits to street-view and aerial images to raise the score of a target urban indicator like lively, safe, and greenery. The framework is easy to understand and straightforward: a planner picks the region to edit, a policy agent sets constraints, segmentation and in-painting agents produce the edit, and each has its own quality check, a black-box indicator model and LLM then score the candidates. The authors also introduce an 8-task benchmark across two modalities and compare against adapted counterfactual and urban-perception editing baselines. According to the LLM and a small expert study, VIDA-GEO produces more realistic and more policy-consistent edits, although its raw indicator gains are often smaller than those of the baselines.
Critical strengths
The problem framing is valuable. Most work in this area stops at recognizing or scoring urban scenes. Instead, a “feasibility-centric” framework is a useful step toward actual decision support. The idea of pairing an indicator model with a controllable editor as a kind of implicit digital twin is a nice way to put it. The pipeline is modular and easy to follow. Because edits are tied to specific regions and come with textual rationales, a planner could actually inspect and question them. That is more useful than a global restyling of the whole image.
The quality-control agents for masks and edits are a sensible design choice, and the ablations suggest they matter. The benchmark covers two modalities and a reasonable set of baselines, and the call budget is equalized across methods. The expert study is a welcome addition on top of the LLM results.
Critical weaknesses
The whole setup assumes the black-box scorers behave sensibly on counterfactual inputs. These models are known to pick up shortcuts (more green means safer, for instance). The main paper should show some stress test, such as color- or texture-only edits, copy-paste artifacts, or edits with a known expected effect.
The constraints come from the LLM’s general knowledge, not from local zoning or regulations. That’s fine for a research prototype, but “policy alignment” should then be described more carefully. An example where city-specific rules change the allowed actions would make the claim much stronger.
Both generation and evaluation depend on proprietary models. The authors themselves report that quality drops with open-source substitutes. There’s also no reporting of runtime, number of API calls, or cost per image and per epoch. That makes it hard to asses whether this could run at anything close to city scale. In general, a table summarizing LLMs’ parameters is also missing.
Mahalanobis distance needs a proper definition: which feature backbone, which reference set, whole image or masked region, and how it correlates with human or a judge ratings. For localized edits, it seems not being very sensitive. The expert study also needs more details: number of pairs per task, blinding, inter-rater agreement, and how ties were handled.
A proper literature review that includes recent works in multidisciplinary journals regarding how multimodal models are used to evaluate urban scenarios is needed.
Questions for authors
See weaknesses above.