- Rating
- 2
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Parallel Instance Decoding (PID), a two-stage generative localization framework for aerial imagery. PID first autoregressively predicts a variable-length set of point addresses and then predicts all address-conditioned boxes in one packed forward pass using instance-isolated attention. It is evaluated on category-conditioned detection, referring-expression grounding, and pointing, with controlled comparisons against a finetuned direct-box baseline and explicit latency/throughput analysis.
Critical strengths
- The controlled comparison is unusually careful: the direct-box baseline uses the same training data, curriculum, trainable components, and checkpoint-selection procedure, and the paper additionally provides class-isolated controls.
- The paper evaluates both accuracy and complete-response efficiency, and the density-stratified analysis is aligned with the claimed motivation that parallel decoding should matter most in dense aerial scenes.
Critical weaknesses
- The instance-isolated attention mask is the core mechanism that enables independent parallel box decoding, yet it is never directly ablated. Without a packed-decoding variant that allows cross-instance attention, the paper does not show that masking itself is responsible for the accuracy gain.
- The architectural contribution is confounded with additional point-center supervision. PID receives explicit address targets while the direct-box baseline does not; the paper itself acknowledges that the effect of point supervision cannot be fully separated from instance addressing. A direct-box baseline with the same auxiliary point objective is necessary.
- The efficiency claim is not fully clean. The final detection numbers use a rotary-stabilized implementation, whereas the reported timing was measured before that numerical change. Moreover, specialized detectors remain dramatically faster in absolute throughput, so the claimed efficiency advantage is primarily within autoregressive VLM-style localizers rather than localization methods in general.
- The 'open-vocabulary' framing is not validated by the main detection protocol. Queries only contain categories known to be annotated in each image, so the evaluation does not test absent-category rejection or unseen-category generalization. The current evidence supports category-conditioned localization, not unrestricted open-vocabulary detection.
- There is an unresolved train/evaluation contamination risk. The training pool contains DIOR/DOTA-family sources, while the authors only remove exact image overlaps and explicitly state that geographic independence between nearby aerial tiles is not guaranteed. A near-duplicate/geospatial leakage analysis is needed before interpreting the gains as clean generalization.
Questions for authors
- What happens if box segments are allowed to attend to other boxes during the packed pass? Please provide a direct ablation of the instance-isolated attention mask.
- Can the direct-box baseline be trained with the same auxiliary point-center supervision to isolate the effect of PID's architecture from the extra supervision signal?
- Can the authors report latency/throughput using the exact rotary-stabilized implementation used for the final accuracy results?
- Can the authors provide quantitative absent-category and unseen-category evaluation, and a nearest-neighbor/geographic-overlap analysis between training and evaluation imagery?
4 comments
W3, Q3: Efficiency with the rotary-stabilized implementation
The Table 4 timings were recorded with the rotary-stabilized implementation, in the same evaluation run that gives the Table 1 accuracy. The run log shows the FP32 rotary cache installed in all 36 rotary modules before any prefill, and the same outputs give both tables: the ratio of correct to valid boxes per second in Table 4 (5.50 / 6.99 on DIOR and 8.33 / 13.30 on DOTAv2) matches, up to rounding, the micro-precision of the Table 1 outputs (2,473 / 3,146 and 16,158 / 25,785). The statements in Sec. 4.3, App. B.3 and the caption of Table 18 (Table 15 in the submission) were incorrect and are corrected in the revision. For convenience, we repeat the comparison with the direct-box baseline, which uses the same backbone, data and training:
| DIOR [email protected] | DIOR s/image | DIOR Correct boxes/s | DOTAv2 [email protected] | DOTAv2 s/image | DOTAv2 Correct boxes/s | |
|---|---|---|---|---|---|---|
| LA-3B (FT)† | 73.94 | 0.990 | 2.90 | 59.62 | 3.930 | 3.27 |
| PID (ours) | 82.24 | 0.768 | 5.50 | 66.90 | 2.218 | 8.33 |
| Speed-up | 1.29× | 1.77× |
PID is both faster and more accurate. It reduces latency by 22.4% on DIOR and 43.6% on DOTAv2 and gives 1.90× and 2.55× more correct boxes per second. The advantage grows with scene density, as expected from Eq. 3. With the same addresses, predicting all boxes in one pass is 3.0× (DIOR) and 5.9× (DOTAv2) faster than predicting them one at a time, with the same accuracy (App. D.3, Table 19B).
We agree that specialized detectors are faster in absolute time (4–12×, Table 18), and the revision limits our efficiency claim to generative VLM-based localizers. Even among these, PID is not the fastest everywhere: on DOTAv2, the pretrained LA-3B has slightly lower latency (1.992 vs. 2.218 s, Table 18), but PID produces 2.8× more correct boxes per second (8.33 vs. 3.00). The statement in Sec. 4.3 that PID has the lowest latency on both datasets is replaced by the statement that it has the highest throughput among generative VLMs. However, the specialized detectors are not evaluated here on referring-expression grounding or pointing, which PID handles with the same model. In accuracy, PID is far above Grounding DINO (82.24 vs. 27.48 on DIOR and 66.90 vs. 17.17 on DOTAv2) and above LAE-DINO on DIOR at stricter thresholds ([email protected]: 69.83 vs. 58.98; mF1: 65.91 vs. 54.75). LAE-DINO, which is pretrained on the large LAE-1M dataset, remains stronger on DOTAv2 (Table 1).
W4, Q4 (a): Absent-category and unseen-category evaluation
We agree that the main protocol measures category-conditioned localization. It follows the annotated-category detection setting of Rex-Omni (App. C.1), and the revision states this scope and replaces "open vocabulary query" with "class name or phrase" (Abstract and Sec. 1). To test absent categories, we asked for all 20 DIOR and 18 DOTAv2 categories in every image, so every prediction for a category that is not in the image is a false positive:
| Model (all categories requested, same run) | [email protected] DIOR | [email protected] DOTAv2 | Absent-category predictions DIOR | Absent-category predictions DOTAv2 |
|---|---|---|---|---|
| LA-3B (FT)† | 69.83 | 55.12 | 4,313 | 27,235 |
| LA-3B (FT, class-iso.)§ | 73.23 | 59.96 | 153 | 2,318 |
| PID (ours) | 66.56 | 56.54 | 2,615 | 15,424 |
The class-isolated model is the most accurate in this setting and rejects absent categories best, and on DIOR PID is also below LA-3B (FT)†. PID makes 39% (DIOR) and 43% (DOTAv2) fewer absent-category predictions than LA-3B (FT)† and is more accurate on DOTAv2. These errors start in the address step: absent categories still receive addresses, and the box step does not reject all of them. Since predicting the boxes in one pass or one at a time gives the same result, they are not caused by parallel box prediction.
We additionally test robustness to new wordings of the seen categories. Every category is requested with one fixed new wording; the wordings were generated with an LLM assistant, checked with a case-insensitive substring search over all training records, fixed before any model was evaluated, and are identical for all models (full list in App. D.6, Table 23). The evaluation labels are unchanged.
| Model | DIOR Original | DIOR New wording | DIOR Retained | DOTAv2 Original | DOTAv2 New wording | DOTAv2 Retained |
|---|---|---|---|---|---|---|
| LA-3B (FT)† | 73.94 | 61.66 | 83% | 59.62 | 40.62 | 68% |
| LA-3B (FT, class-iso.)§ | 75.45 | 50.60 | 67% | 61.66 | 30.46 | 49% |
| PID (ours) | 82.24 | 71.21 | 87% | 66.90 | 48.51 | 73% |
All values are [email protected]. Examples: storage tank → "cylindrical oil silo", helicopter → "rotorcraft", chimney → "industrial smokestack". "Original" is from Tables 1 and 5A.
PID is the most accurate model with the new wordings and keeps the largest part of its accuracy. This tests robustness to new wordings, not generalization to unseen categories. Both analyses are added in App. D.6 (Tables 21–23), with a pointer in Sec. 4.1.
W5, Q4 (b): Near-duplicate and geographic-overlap analysis
The overlap analysis leads to three main findings. First, on the 332 DIOR test images that share no ground area with any training image, PID keeps its full gain (+6.7 over the class-isolated model and +8.1 over LA-3B (FT)†). Second, most of the DOTAv2 overlap comes from the LAE DOTAv2 benchmark split itself: its training and test tiles are cut from the same source images, and 523 of the 567 source images of the test tiles also have tiles in its training file. The DOTAv2 subset of LAE-1M is sampled from this training file and contains tiles of 517 of these 567 source images, so the overlap is a property of the benchmark split and also affects models pretrained on LAE-1M, such as LAE-DINO. Third, the gain of PID remains large when scenes of similar density are compared: on the DOTAv2 images with less than 25% overlap, PID's gain over LA-3B (FT)† is +10.5 in both the 6–20 and the >20 object groups, comparable to its gain in the same groups on the overlapping images (+13.8 and +11.6). The details are given below.
For every DIOR and DOTAv2 test image, we searched all 106,318 training images using CLIP ViT-L/14 similarity (full image and five crops) and perceptual hashing. We then checked each candidate with SIFT + RANSAC homography and pixel-level correlation. This check accepts 0 of 4,000 random image pairs from the same dataset. Since DIOR and DOTA have no geographic coordinates, we use a verified homography as a proxy for showing the same ground area.
| Test images that share ground area with a training image | DIOR | DOTAv2 |
|---|---|---|
| Any overlap | 254 / 586 | 792 / 874 |
| Overlap covering ≥ 25% of the test image | 247 / 586 | 603 / 874 |
| Overlap covering ≥ 50% of the test image | 18 / 586 | 393 / 874 |
All compared models are trained on the same data, so overlap affects them all, although not necessarily by the same amount. To check this, we removed the overlapping test images and evaluated again:
| [email protected] | Images | LA-3B (FT)† | LA-3B (FT, class-iso.)§ | PID | PID gain (vs. class-iso. / vs. LA-3B (FT)†) |
|---|---|---|---|---|---|
| DIOR, all images | 586 | 73.94 | 75.45 | 82.24 | +6.79 / +8.30 |
| DIOR, without any overlap | 332 | 72.31 | 73.73 | 80.39 | +6.66 / +8.08 |
| DOTAv2, all images | 874 | 59.62 | 61.66 | 66.90 | +5.24 / +7.28 |
| DOTAv2, without ≥ 50% overlap | 481 | 61.14 | 61.85 | 66.17 | +4.32 / +5.03 |
| DOTAv2, without ≥ 25% overlap | 271 | 63.51 | 63.48 | 65.89 | +2.41 / +2.38 |
On DOTAv2, PID stays the best model after removing images with ≥50% or ≥25% overlap, although the gap becomes smaller and is no longer significant at the ≥25% threshold. The low-overlap DOTAv2 images are also much sparser (median 5 objects, compared with 9 over all DOTAv2 images). In the sparsest group (1–5 objects) of these images, LA-3B (FT)† is ahead by 5.0 points, so together with the third finding above, the smaller overall gain is mainly explained by the larger share of sparse images. Only 82 DOTAv2 images have no overlap at all. On this small subset, LA-3B (FT)† scores 70.29, PID 68.35 and the class-isolated model 67.01, but the difference to PID is not significant. The overlap-filtered DOTAv2 and overlap-free DIOR numbers are reported next to the main results (Sec. 4.2), the full analysis is in App. A.3 (Table 10), and we will release the overlap lists.
In summary, each of the five weaknesses is now addressed in the revised manuscript, either with a new experiment (mask ablation, point-supervised baseline, timing provenance, absent categories and new wordings, overlap analysis) or with a narrowed claim (efficiency among generative VLM localizers, category-conditioned localization). We hope this resolves the reviewer's concerns and would be grateful if the reviewer would reconsider the score.
We thank the reviewer for their constructive comments. We have uploaded a revised manuscript; all changes are highlighted in blue. All results below use the same rotary-stabilized implementation as the final results in the paper.
W1, Q1: Ablation of the instance-isolated mask
In the parallel box pass, all boxes are predicted together in one forward pass, so no box sees the decoded coordinates of another box. Without the mask, however, the tokens of each box could still attend to the segments of the other boxes. The instance-isolated mask blocks this, so each box depends only on the image prefix and its own address (Sec. 3.2). We test the mask in two ways. (a) We train PID with cross-box attention: each box segment may also attend to the reference–address prompts of all other box segments. Attention between the geometry positions of different boxes cannot be trained this way, because during training these positions hold the target coordinates and would leak the answers; variant (b) covers this case. Everything else in (a), including the point targets and the loss, is the same as for PID. (b) We take the final PID model, trained with the mask, and open the mask only at test time, so that each box segment attends to all tokens of the earlier segments or of all other segments.
We report (a) in a cumulative ablation that adds the parts of PID one at a time to the class-isolated direct-box model LA-3B (FT, class-iso.)§ (Table 5A). The second row is the direct-box model with the same point supervision as PID, which answers W2:
The main evidence is (a): even when the model is trained with cross-box attention, the instance-isolated mask adds +2.9 points on DIOR and +3.5 points on DOTAv2 on top of addressing. Attending to other boxes therefore does not help box prediction, and it hurts more in the denser DOTAv2 scenes. Variant (b) is a stress test rather than a clean ablation: the model was never trained with cross-box attention, and all box segments share the same position indices (App. B.2). Opening the mask at test time reduces [email protected] by up to 45.2 points on DIOR and 52.4 points on DOTAv2, which shows that the trained model relies on the mask. The mask also makes each box independent of the number and order of the other objects. This is why all boxes can be predicted in one pass with the same accuracy as one at a time ([email protected] differs by only 0.06 on DIOR and 0.00 on DOTAv2). This is added in Sec. 4.4 (Table 6) and App. D.5 (Table 20).
W2, Q2: Direct-box baseline with the same point supervision
We trained this baseline (second row of the table above). It is the class-isolated direct-box model, which is stronger than LA-3B (FT)† and makes the same per-category requests as PID. We add the point targets of PID as separate point requests in the same training episodes, next to the per-category box requests and with the same image prefix. There is no extra head, and no point is written before a box. The two losses have equal weights, as in Eq. 4, and data, curriculum and checkpoint selection are the same as for PID. Its supervision is therefore the same as PID; the remaining difference is that PID predicts each box from its own address.
The point loss helps on DIOR (+2.3) but not on DOTAv2 (−1.0), where the point-supervised model returns about 1.7× as many boxes with the same number of correct boxes, i.e., more false positives. PID remains 4.5 / 6.2 points above this control: predicting boxes from addresses adds 1.5 / 2.7 points, and the instance-isolated mask a further 2.9 / 3.5. Most of the gain therefore comes from predicting each box from its own address with the isolated mask, not from the extra supervision. This is added in Sec. 4.4 (Table 6) and App. D.5.