- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes LAM++, an improved version of the LocateAnything Model (LAM) for language-guided object localization, focusing on dense scenes and small objects. The main contributions include: constructing a dataset of over 30 million localization examples, emphasizing challenging cases, using high-resolution images and single-query training to improve localization accuracy, applying GRPO with a localization-specific reward to improve bounding-box predictions, introducing geometry-aware speculative decoding (GeoSpec) that uses fast MTP predictions as drafts and AR predictions for verification. Experiments across 11 benchmarks show improvements over LAM and other localization models, particularly in dense scenes. GeoSpec achieves approximately 68% higher box-decoding throughput than AR decoding with small accuracy degradation.
Critical strengths
- Failure-driven approach: The authors identify specific weaknesses of LAM in dense scenes and small-object detection, then design their training strategy to address these failures.
- Strong empirical results: LAM++ substantially improves over the original LAM across different benchmarks, with particularly impressive improvements on Dense200.
- Well-engineered paper: The paper combines large-scale data construction, high-resolution training, GRPO, and efficient decoding into a well-engineered localization framework.
- Broad experimental evaluation: The experiments cover 11 benchmarks across object detection, dense localization, referring grounding, reasoning grounding, GUI grounding, and other domains.
- Simple and effective GeoSpec: GeoSpec is well-motivated, straightforward to implement, and achieves significant decoding speed improvements with minimal accuracy loss.
Critical weaknesses
- Limited methodological novelty: Most accuracy improvements rely on established techniques. High-resolution and single-query training are straightforward, failure-focused data selection is not conceptually new, and GRPO was previously used for generative localization in Rex-Omni. GeoSpec appears to be the main distinct algorithmic contribution.
- Unclear contribution of additional computation: The model uses substantially higher image resolution, more training data, and extensive post-training. Comparisons with LAM are not fully matched in terms of resolution, training data, and computational budget.
- Limited computational efficiency analysis: The paper emphasizes decoding throughput but lacks a sufficiently detailed comparison of overall inference latency, memory, and computational costs, particularly considering the higher input resolution.
- Dependence on strong teacher models: The pseudo-labeling pipeline relies heavily on models such as SAM3 and Rex-Omni, raising questions about data quality, scalability, and how much of the improvement comes from stronger supervision.
- Comparison with SAM3: The paper emphasizes outperforming SAM3, but the models have different architectures and intended capabilities. Some evaluated tasks may not be well suited to SAM3's query formulation, making parts of the comparison less informative.
- Heuristic GeoSpec verification: GeoSpec uses a fixed coordinate tolerance, although the effect of coordinate errors depends on object size. An IoU-based or object-scale-aware acceptance criterion may be more appropriate, particularly for small objects.
Questions for authors
-
Q1. Compute-matched comparison: How much of the improvement over LAM comes from higher input resolution, additional training data, and training compute? Can the authors provide a more controlled comparison?
-
Q2. GeoSpec generalization: Can GeoSpec be applied to other autoregressive localization models that do not already have LAM's MTP policy? How much additional training or architectural modification would be required?
-
Q3. Data contamination: How do the authors ensure that the large-scale web-derived training data does not overlap with evaluation benchmarks?
-
Q4. GeoSpec acceptance: Have the authors considered IoU-based or object-size-aware acceptance criteria instead of a fixed coordinate tolerance? How does the current criterion affect small-object localization?
9 comments
W1: Methodological novelty
We agree that high-resolution processing, single-query training, failure-focused selection, and GRPO are established, and we do not claim them as our inventions. Our contribution is the systematic diagnosis that motivates them, plus two results we believe are new:
- A quantified failure diagnosis that drives the recipe. LAM trails SAM3 by only 0.59 mF1 on long-tail LVIS queries but by 23.51 on dense queries (Table A8). The recipe is built against that measurement, and each component is then ablated separately (Part 2).
- Bidirectional cross-policy transfer. Optimizing only the AR policy improves MTP (62.48→63.85 LVIS mF1), and optimizing only MTP improves AR (64.16→65.71), even though neither receives the other's loss. MTP-GRPO, however, shifts both policies to a more conservative operating point (dense AR recall 38.24→36.09, vs. 42.01 for AR-GRPO). This characterizes RL transfer between decoding policies that share parameters.
- GeoSpec, which the reviewer already identifies as the main algorithmic contribution. We would add that its benefit is not only speed: accepting MTP's jointly predicted coordinates within a tolerance measurably improves box geometry, and the gain grows with the IoU threshold (LVIS: +1.0 F1 at IoU 0.5, +2.2 F1 at IoU 0.9).
W2+Q1: Compute-matched comparison with LAM
This is a fair concern, and we answer it at two levels.
(1) Inference Compute, Fully Matched (Table 1)
The input setting can be changed without any retraining, so we evaluated the original LAM under all four input settings. Mean mF1 over the 11 benchmarks:
| Input Setting | LAM AR | LAM++ GeoSpec | Δ | LAM++ Better On |
|---|---|---|---|---|
| Multi-query, original res. (LAM default) | 46.33 | 50.16 | +3.84 | 8 / 11 |
| Single-query, original res. | 47.83 | 54.56 | +6.74 | 11 / 11 |
| Multi-query, high res. | 48.56 | 53.21 | +4.65 | 9 / 11 |
| Single-query, high res. (LAM++ default) | 49.24 | 57.62 | +8.38 | 11 / 11 |
Giving LAM the same high-resolution input closes only 2.24 mF1 of the 11.29 mF1 gap. Under every matched setting, including LAM's own default format which LAM++ was never trained for, LAM++ remains ahead. The advantage therefore does not come from spending more inference compute.
(2) Training Data and Compute (Table A8)
Each ablation step is matched on everything except the factor under study: RSL vs. SSL use the same 450K budget; single-query uses the same data as SSL; high resolution uses the same data and query format. This isolates:
- +5.92 mF1 for the silver data;
- +0.97 mF1 for single-query (+3.82 for MTP);
- +10.02 mF1 for resolution;
- +2.05 mF1 for AR-GRPO.
Rebuttal Overview
We sincerely thank the reviewer for the careful reading and constructive suggestions. Such thoughtful feedback is particularly encouraging in today's fast-moving research environment and helps us further improve the paper and make a more meaningful contribution to the research community.
Before the point-by-point response, we briefly summarize the key revisions (see the General Response for details):