- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
I am not an expert in this domain. Based on my understanding, this paper presents FlatLands, a benchmark for completing unseen floor regions from a single egocentric view. It combines six indoor datasets into approximately 270,000 observations and compares eleven methods, including deterministic and generative models. The evaluation covers map completion, transfer to RGB-derived observations, and offline path planning. The main contribution is the dataset and evaluation framework for studying multiple plausible floor maps.
Critical strengths
- The paper addresses an important problem: a single view provides incomplete information about the surrounding environment. Representing several possible maps is well motivated, particularly when incorrect predictions could affect navigation.
- The shared representation and evaluation protocol allow different completion methods to be compared across several datasets. This could provide a useful benchmark for studying completion quality and its relationship to planning performance.
Critical weaknesses
-
The planning comparison does not clearly isolate the benefit of multiple maps. Generative models use probability-based traversal costs, while U-Net uses a thresholded map with uniform costs. Therefore, the improvement may partly come from the planning strategy rather than the generated maps.
-
The evidence for recovering multiple plausible layouts is limited. In Table 15, All-Floor achieves the same coverage as the generative methods and a lower symmetric distance. This example does not convincingly demonstrate the advantage of generating multiple solutions.
Questions for authors
- Would the planning improvement remain if U-Net’s predicted floor probabilities were used with the same traversal costs and cell-admission rules as the generative methods?
- Can the authors provide additional evidence that the generated samples capture distinct plausible layouts? In Table 15, a single All-Floor prediction already matches the reported coverage of the generative methods.
7 comments
Weakness 2: Whether Table 15 demonstrates distinct layouts
The evidence for recovering multiple plausible layouts is limited.
Table 15 illustrates coverage on one scene at a 0.10 Jaccard-distance tolerance. The set-level evidence is in Question 2.
In Table 15, All-Floor achieves the same coverage as the generative methods and a lower symmetric distance.
Four of the five targets lie within 0.10 of All-Floor, which accounts for its coverage and symmetric distance:
| Method | Targets covered out of 5 | Symmetric distance ↓ |
|---|---|---|
| All-Floor | 4 | 0.051 |
| Diffusion | 4 | 0.054 |
| Flow Matching | 4 | 0.054 |
| FM+XAttn | 4 | 0.069 |
The disagreement-cell test and the repeated-sample comparison in Question 2 do not depend on this tolerance, and we will add both beside Table 15.
This example does not convincingly demonstrate the advantage of generating multiple solutions.
On the cells where the targets disagree, any single map reused for both targets scores exactly 50%, while sample pairs from each generator score 60.9 to 66.2% (Question 2). The sample-count control (Weakness 1) shows the same advantage in planning, where four maps beat one for every generator.
Question 1: Does the improvement remain when U-Net uses the same costs and admission rules?
Would the planning improvement remain if U-Net's predicted floor probabilities were used with the same traversal costs and cell-admission rules as the generative methods?
Yes, the improvement from probability-weighted costs remains. With identical allowed cells and costs, U-Net probabilities reach 84.75% success, FM+XAttn frequencies 83.28%, and unit costs 81.20%. Multiple maps add a separate gain: four maps beat one for every generator (Weakness 1).
Protocol: 4,000 fixed queries from 500 geometry-derived observations. All methods share observed floor plus the hidden valid cells that at least one FM+XAttn sample labels floor, and the goal is always admitted. Search is eight-connected Dijkstra without diagonal corner cutting. Observed floor costs 1, and hidden cells cost , where is a U-Net probability or FM+XAttn frequency; each move costs its length times the mean endpoint cost. Success requires the route to lie entirely on valid reference floor.
| Cost source | Overall success | Success among returned routes |
|---|---|---|
| Unit costs | 81.20% | 84.83% |
| U-Net probabilities | 84.75% | 88.53% |
| FM+XAttn | 83.28% | 86.99% |
Thank you for this question: it helped us establish that richer per-cell probabilities help in any case, whether they come from deterministic model's output probabilities or from direct outputs of generative models. This will most definitely be presented in the extended main paper.
Weakness 1: Separating the effect of the maps from the planning strategy
Thank you for the thoughtful review and for recognising the motivation for representing several possible maps. The comparison you proposed helped us isolate the benefit of multiple maps.
To isolate it, we fix the queries (4,000), the graph (observed floor plus all hidden valid cells), and the costs, and vary only the number of maps each generator uses, averaging over all subsets of its four saved maps. This graph admits more cells than the method comparison in Question 1.
Four FM+XAttn maps raise success by 1.47 percentage points over one (paired 95% interval over physical environments 0.81 to 2.21).
The two controls separate the effects. With the same allowed cells and costs, both U-Net probabilities (84.75%) and FM+XAttn frequencies (83.28%) improve over unit costs (81.20%; Question 1); with everything else fixed, four maps beat one (above). Retaining ambiguity, as per-cell probabilities or as multiple maps, improves planning.
We will add both controls beside the planning results, with their protocols in the appendix.