acceptodds
Under review as a conference paper at ICLR 2027

ROOMER: Learning What to See and When to Stop for Room Re-identification

Abstract

Modern room re-identification pipelines extract increasingly rich global, local, fine-grained, and geometric evidence, but still process every query through a rigid cascade. This uniform execution wastes computation on already-resolved queries and can even overturn correct decisions when later evidence is noisy. We present Roomer, a multi-turn vision-language agent that adaptively allocates test-time evidence, deciding at each turn whether to commit, verify candidates geometrically, or inspect them semantically. To equip the agent with a reliable semantic eye, we introduce Sight-OPSD, which uses training-only regional and geometric teaching views to teach a 4B raw-image Inspector what to see: which cross-view evidence supports or refutes shared room identity. The resulting Inspector requires no privileged inputs at inference and also serves as Roomer’s policy backbone. We train the Router with cost-efficient multi-turn group-relative credit assignment. Beyond trajectory-level Outcome Credit for final correctness, localized Commitment Credit supervises when to stop, while Probe Credit uses evidence displacement to assign credit to semantic inspections that actually improve the evidence state. This localized supervision mitigates an afraid-to-commit collapse in which the policy keeps acquiring evidence instead of answering, while discouraging uninformative use of the expensive semantic Inspector. Roomer improves both recognition and evidence efficiency: Sight-OPSD raises the 4B Inspector’s query-weighted balanced accuracy from 87.80% to 91.12%, exceeding the 397B Qwen3.5 model by 1.46 points, while the full agent reaches 88.79% Recall@1 on AirRoom-Contested, 10.27 points above the fixed AirRoom Full pipeline. On the natural AirRoom distribution, Roomer further reaches 96.46% Recall@1, a 2.19-point improvement over AirRoom Full. Compared with correctness-only routing, localized credit further improves contested Recall@1 by 1.59 points while using 38.8% less tool budget—a Pareto gain showing that Roomer can perform better by looking less. The code will be made publicly available upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
room re-identification, visual agents, on-policy self-distillation, multi-turn reinforcement learning, multi-turn credit assignment

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_VJhfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ROOMER, a multi-turn vision-language agent for room re-identification. Instead of a fixed coarse-to-fine pipeline such as AirRoom, a learned Router reads a shortlist from a frozen global retriever and decides at each turn whether to commit to a candidate, run a geometric verifier, or semantically inspect a candidate with an Inspector. The Inspector is trained with Sight-OPSD, an on-policy self-distillation scheme in which a teacher sees training-only regional and geometric views while the student sees only the raw image pair. The Router is a LoRA adapter on the frozen Inspector backbone, trained with grouped multi-turn RL that combines trajectory-level Outcome Credit with localized Commitment and Probe Credits, which supervise when to stop and which candidate to inspect. The authors also introduce AirRoom-Contested, a diagnostic subset of ambiguous queries. ROOMER improves over strong fixed pipelines on both the contested subset and the natural distribution while using less runtime, and the distilled 4B Inspector outperforms a much larger general-purpose model on pair verification.

Critical strengths
  • The method improves over strong baselines, including AirRoom+SLiM, on both the known benchmarks and on the ambiguous queries subset. A distilled 4B Inspector outperforming a general-purpose 397B model is an impressive result.
  • The available ablations (Tables 3 and 4) are informative and appreciated, and the routing behavior table (Table 5) gives useful insight into how the Router allocates evidence.
Critical weaknesses
  1. Efficiency claims are not carefully supported.
  • The motivating claim that later noisy evidence overturns correct decisions in fixed pipelines is not quantified (see Question 3).
  • Table 5 does not state the hardware or the inference engine (optimized or plain PyTorch). The compared systems also spend their time on different kinds of computation (autoregressive VLM generation versus feature extraction and matching), so total latency is difficult to interpret without a per-component breakdown (direct, geometry, inspection) for ROOMER and for AirRoom with and without SLiM. The "38.8% less tool budget" is expressed in hand-set abstract cost units (Appendix E) that are not defined in the main text and are not wall-clock time.
  1. Probe Credit relies on hand-designed components that are not ablated. Eq. 7 introduces a lexicographic ordering with a verdict tier, and Eq. 8 defines displacement as the sum of a rank change and a tier change. Under this ordering a positive verdict changes both quantities, so the signal is partly double counted. The paper gives one sentence of justification. Table 4 ablates the credits as a whole but not these internal choices.
Questions for authors
  1. Why is the Router built on the Stage-I Inspector backbone rather than a separate model? Does the verification-oriented Stage-I training help or hurt routing, for example compared with a Router on the unadapted Qwen3.5-4B?
  2. Was RL training stopped at only 120 steps because performance had converged, or does it saturate or degrade with longer training?
  3. The paper motivates adaptive routing by noting that later, noisier evidence can overturn correct earlier decisions in fixed pipelines. Could the authors quantify this, for example the fraction of queries where AirRoom's Local or Fine-Grained stage changes a correct earlier top-1 into an incorrect one, and the corresponding rate for ROOMER (final answer versus the retrieval and geometric leaders)?
  4. In Table 1, the natural-distribution AnyLoc row matches the values reported in the original AirRoom paper (its Accuracy column) on all four datasets, whereas other baselines (e.g., AirRoom Full, DINOv2) differ. It is also unclear why AirRoom Global differs substantially from AnyLoc on the natural distribution (e.g., Gibson 93.45 vs. 85.85) while the two nearly coincide on AirRoom-Contested. Could the authors clarify whether all baselines were re-evaluated?
  5. The abstract uses several terms before defining them (Router, Sight-OPSD, evidence displacement, "afraid-to-commit collapse"). Could the authors briefly define or describe each term on first use in the abstract?
  6. The abstract reports gains over AirRoom Full and over a 397B general-purpose model. Could the authors also report the gains over the strongest task-specific baselines in Table 1 (AirRoom+SLiM, SAGE)?
  7. Could the authors define T_i, K_i and β in Eqs. 4 to 6?
  8. In Eq. 8, tier is written as a function of c* while the rank terms are written as ρ. Could the authors clarify and unify this notation?
  9. Could the authors state in the main text that each candidate is inspected at most once and geometry is called at most once, and clarify what "multi-turn" means under this constraint?
  10. Could the authors use a consistent capitalization of ROOMER across sections?
  11. Could the authors show qualitative examples of how the teacher's token-level predictions on the student's responses change once the regional or geometric view is provided?
3 comments

We sincerely thank the reviewer for the thoughtful and constructive feedback. Your comments have been very helpful in improving the clarity, rigor, and completeness of our manuscript. We have carefully addressed the concerns and highlighted the corresponding revisions in blue in the updated manuscript for ease of review.

Q1: Shared Inspector Backbone for Routing

Our primary motivation is parameter efficiency, not an assumed transfer benefit from verification training. We reuse the frozen 4B Inspector backbone and train only a lightweight LoRA adapter for routing; removing the adapter preserves the original verification capability. We still conducted a new experiment: training the router starting from a completely new initialization of qwen3.5-4B + LoRA, and observed comparable performance, suggesting that Stage-I adaptation is not essential for initial routing competence.

Q2: Justification for the 120-Step Checkpoint

We selected the step-120 checkpoint based on held-out development performance. Dev R@1 remains stable around 80% through step 120 but drops to approximately 76.5% at step 140. Meanwhile, declining policy entropy and group-advantage magnitude indicate reduced exploration and weaker optimization signals. Thus, step 120 balances training stability and generalization while avoiding late-stage performance degradation.

Q3: Over-Processing and Runtime Efficiency

We have added a paired transition analysis in Sec. 4.5 and Appendix D.3 (Table 9). AirRoom's Global-to-Local and Local-to-Fine-Grained stages overturn 294 (7.74%) and 380 (9.40%) previously correct predictions, respectively. ROOMER correctly resolves 279/294 (94.90%) and 278/380 (73.16%) of these cases through adaptive stopping and selective verification. On the full AirRoom-Contested evaluation set, ROOMER achieves a lower average wall-clock latency of 0.543 s/query, compared with 0.676 s/query for AirRoom Full, measured on NVIDIA A100 80GB GPUs. Despite the autoregressive overhead of Router generation, ROOMER invokes semantic inspection for only 18.8% of contested queries, while 12.0% are resolved directly and 69.1% stop after geometric verification. This demonstrates that adaptive evidence acquisition and timely stopping can offset the generation overhead. We will release the wall-clock benchmarking code upon acceptance to facilitate reproducibility.

Q4: Baseline Evaluation Consistency

We thank the reviewer for pointing out this issue. Upon re-examination, we found that the AnyLoc results on the natural distribution were inadvertently retained from the original AirRoom paper, rather than reflecting a separate evaluation in our submission. In our implementation, AirRoom Global and AnyLoc use the same global retrieval method. We have therefore removed the redundant AnyLoc entry and merged the two into a single AirRoom Global (AnyLoc) entry in the revised Table 1, reporting the results obtained under our evaluation protocol.

We clarify that all reported methods were evaluated under the same scene-disjoint protocol, with identical test queries and galleries within each setting, as detailed in Appendix A. We have also carefully rechecked the remaining baseline results and confirmed their correctness and consistency with our evaluation protocol. Upon acceptance, we will release the model weights and complete evaluation protocol to facilitate independent reproduction and verification by the research community.

Q5, Q6 & Q10: Abstract Clarity, Baseline Comparisons, and Naming Consistency

We have revised the abstract to define Sight-OPSD, Inspector, and Router at first mention (Q5). We also explicitly report ROOMER's improvements over the strongest task-specific baselines. Finally, we have standardized the method name as ROOMER throughout the paper (Q10). Please see the revised Abstract (p. 1).

Q7: Definitions of TiT_i, Ki\mathcal{K}_i, and β\beta

We have clarified these notations in Sec. 3.3 (Eqs. 4–6). TiT_i denotes the terminal decision turn, Ki\mathcal{K}_i specifies the turns receiving Commitment Credit (terminal commitment or unnecessary subsequent tool calls), and β=0.5\beta=0.5 controls its strength. These clarifications do not change the underlying method.

Q8: Notation Consistency in Eq. (8)

We have unified the notation in Sec. 3.3 (Eq. 8) by explicitly writing ρi,t±(c⋆)\rho^\pm_{i,t}(c^\star). Both rank and tier displacement are evaluated for the same ground-truth candidate c⋆c^\star. This is a notational clarification without changing the computation.

Weakness-2: Probe Credit Design and Ablation

We agree that rank and tier displacement are coupled, as tier participates in the lexicographic ranking. However, they provide complementary supervision: rank captures relative-order improvements and penalizes harmful promotions of incorrect candidates, while tier rewards direct positive confirmation even when the ground-truth rank remains unchanged. Their combination is an intentional reward-shaping design, rather than two independent information gains.

To assess the additional value of tier, we added a Rank-only Probe Credit ablation, retaining the tier-aware ranking in Eq. (7) while removing the explicit tier-displacement term in Eq. (8). This isolates the incremental value of the explicit confirmation bonus. Removing tier from both equations would yield zero inspection displacement, since semantic inspection changes neither geometric evidence nor retrieval similarity, leaving candidate ordering unchanged. Such a setting would therefore provide no meaningful inspection supervision.

MethodR@1 ↑Budget ↓Inspect ↓
Outcome-only GRPO87.204.1042.0%
+ Rank-only Probe Credit87.163.5231.4%
+ Probe Credit (Rank+Tier)87.433.7034.0%

Rank-only delivers substantial efficiency gains. Adding tier further improves R@1 from 87.16% to 87.43%, with a modest increase in budget (3.52→3.70) and inspection rate (31.4%→34.0%). This is consistent with tier encouraging additional semantic confirmation beyond immediate ranking gains, yielding an accuracy–cost trade-off.

We have clarified the design in Sec. 3.3 (Eqs. 7–9) and added the ablation and discussion in Sec. 4.4 (Table 4).

Q9: Clarification of Multi-Turn Routing

Multi-turn refers to sequential, state-dependent decisions, not repeated verification of the same candidate. After each tool result, the Router updates its evidence state and chooses whether to inspect another candidate or commit. Appendix G.2 (Table 16) shows a four-turn trajectory and we have clarified this in Sec. 3.3, with tool-call constraints detailed in Appendix C.3.

Q11: Example for Token-Level Distillation

We have added a controlled fixed-prefix diagnostic in Appendix B.4 (Figure 4). Using identical frozen 4B weights, prompts, and a student-generated prefix, privileged regional and geometric contexts reduce the probability of the student's erroneous next token (different) from 20.62% to 10.23% and 3.01%, respectively. Across 269 response positions, explanation-content tokens account for 70.9% and 67.7% of the measured KL divergence. This demonstrates that privileged visual context substantially reshapes token-level predictions, supporting our motivation for token-level distillation.

I thank the authors for the detailed response and the additions, which address most of my concerns. In particular, the transition analysis (Table 9) quantifies how often fixed pipelines overturn correct decisions, the AnyLoc entry in Table 1 has been corrected, and the abstract and notation have been clarified. A few concerns are only partially addressed. The runtime comparison still lacks a per-component breakdown (direct, geometry and inspection for ROOMER, and the stages of AirRoom with and without SLiM), and the abstract still reports the 38.8% budget reduction in hand-set cost units defined only in Appendix E. For Probe Credit, the new rank-only ablation shows that the tier term adds only 0.27 points of R@1 while increasing the budget (87.43% vs. 87.16%, 3.70 vs. 3.52), so the benefit of this design choice remains small. I appreciate that the authors report this trade-off openly. Finally, the Router trained from the unadapted Qwen3.5-4B is described as comparable but is not reported quantitatively.

Overall, I keep my positive assessment and maintain the score of 3.

Reviewer_n4oBOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ROOMER, an adaptive multi-turn vision-language framework for room re-identification that selectively acquires visual evidence to identify previously observed rooms. The method first introduces Sight-OPSD, an on-policy self-distillation approach that transfers regional and geometric knowledge from a privileged teacher to a 4B vision-language Inspector operating on raw images. It then trains a Router using multi-turn reinforcement learning with three credit signals—Outcome, Commitment, and Probe—to determine when to make a final prediction, perform geometric verification, or inspect individual candidates. The authors also construct AirRoom-Contested, a challenging evaluation subset, and conduct experiments across four room re-identification datasets. The results demonstrate improvements in recognition accuracy and inference efficiency compared with existing fixed-pipeline methods.

Critical strengths
  1. Strong empirical performance across multiple datasets. ROOMER achieves 88.79% Recall@1 on AirRoom-Contested, outperforming AirRoom+SLiM by 4.53 percentage points, and 96.46% on the natural distribution. The improvements are particularly meaningful on difficult cases, and the authors compare against several established methods, including SAGE and AirRoom.

  2. Effective knowledge distillation into a relatively small VLM. Sight-OPSD improves the 4B Inspector's balanced accuracy from 87.80% to 91.12%, outperforming substantially larger Qwen models. The ablations show that both regional and geometric teaching views contribute to the improvement. Importantly, these additional inputs are unnecessary during inference.

  3. Convincing accuracy–efficiency improvements from localized RL supervision. Compared with correctness-only GRPO, the proposed Commitment and Probe Credits improve Recall@1 from 87.20% to 88.79% while reducing the tool budget by 38.8%. The ablations separately evaluate both credit mechanisms, providing evidence that they improve the learned routing behavior.

Critical weaknesses
  1. Insufficient justification for the complexity of the proposed framework. ROOMER combines privileged-view distillation, a learned Router, supervised warm-starting, and multi-turn RL with specialized reward mechanisms. However, the paper does not provide a comparison against simpler adaptive alternatives, such as confidence-threshold routing or rule-based escalation from retrieval to geometry and semantic inspection. Consequently, it remains unclear whether a complex RL-based decision policy is necessary to achieve the reported accuracy–efficiency improvements.

  2. Incomplete separation of the contributions of the Inspector and Router. Although the authors provide ablations for Sight-OPSD and localized credits independently, the main end-to-end comparisons do not sufficiently isolate the benefits of a stronger Inspector from those of adaptive routing. For example, evaluating a fixed pipeline equipped with the same Sight-OPSD Inspector, alongside an SFT-only Router, would help establish how much improvement comes specifically from the proposed routing strategy rather than the enhanced underlying verification model.

  3. Limited evidence of robustness and practical generalization. The evaluation relies on four existing room-ReID datasets and a contested subset constructed from those datasets. Although the scene-disjoint protocol is appropriate, the paper does not establish whether the learned routing policy generalizes to substantially different real-world environments, such as buildings with changing furniture, severe occlusion, or different acquisition conditions. Moreover, ROOMER provides no improvement over AirRoom+SLiM on the contested ReplicaReID subset (both achieve 73.72%), suggesting that gains are not uniform across datasets.

  4. Missing Relevant Prior Work and Experimental Comparison. The paper does not cite StructVPR++ (IEEE TPAMI 2025), which explores distilling structural and semantic knowledge for visual place recognition. This is particularly relevant to ROOMER's Sight-OPSD, which similarly aims to transfer structural and visual cues into a model without requiring privileged inputs during inference. Although the approaches differ in their architectures and training objectives, the authors should clearly distinguish their contributions from StructVPR++ and provide a quantitative comparison under a common evaluation protocol, if feasible. Such a comparison would help establish the novelty and effectiveness of the proposed distillation approach.

  5. Figure Clarity and Redundancy. Figure 2 is overly crowded, with too many components, arrows, and small text, making the overall workflow difficult to follow. Additionally, Figures 1 and 2 contain overlapping information, introducing unnecessary redundancy. Simplifying Figure 2 and reducing the overlap between the two figures would improve readability and clarity.

Questions for authors

Please refer to the weakness section,

Reviewer_xWE8OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ROOMER, which learns when to stop or collect more evidence for room re-identification. Sight-OPSD trains a 4B image-pair inspector using regional and geometric views available only to the teacher during training. At test time, the inspector uses raw images. A learned router decides whether to answer, call a geometry verifier, or inspect a candidate.

ROOMER outperforms AirRoom Full and AirRoom with SLiM on both splits, with larger gains on the contested split. Rewards for stopping and useful inspection improve accuracy while reducing tool use.

Critical strengths

The ablation supports the reward design. Commitment credit reduces unnecessary tool use, while adding probe credit further improves accuracy and efficiency. ROOMER is more accurate and faster than AirRoom with the stronger SLiM matcher. This shows that its gains go beyond improving the matcher alone.

Critical weaknesses

The method is evaluated within the AirRoom setup on four datasets. The gain over the stronger baseline is smaller on the natural split and zero on Replica contested. The result shows effective training of a small model, but the comparison with the larger Qwen model alone gives a limited picture.

Questions for authors

how similar are the geometric signals used in training and at test time? Would the router still work well with a different verifier?

2 comments

We appreciate the reviewer's constructive suggestions, which have helped us improve the clarity and completeness of our evaluation.

We acknowledge that the gains are not uniform across datasets. ReplicaReID is particularly challenging: even in the original AirRoom paper, AirRoom underperformed AnyLoc in accuracy on this dataset (87.18% vs. 88.57%). On our contested split, AirRoom's R@1 further drops from 69.83% at its Local stage to 60.58% at Full (−9.25 points), illustrating the risk of noisy later-stage evidence. Without upgrading to the stronger SLiM matcher, ROOMER achieves 73.72% through adaptive evidence acquisition and stopping, improving over AirRoom Full by 13.14 points and matching AirRoom+SLiM (73.72%). On natural ReplicaReID, ROOMER further outperforms AirRoom+SLiM (89.54% vs. 87.72%).

To further assess cross-dataset generalization, we added leave-one-dataset-out (LODO) experiments (Table 5), retraining both the Inspector and Router on the other three datasets. The mean R@1 drop relative to full-data training is 0.84 points on natural and 2.05 points on contested evaluation. LODO ROOMER outperforms AirRoom+SLiM on all four natural datasets and three of four contested datasets, with Replica contested remaining the exception. We acknowledge this limitation while noting the overall transferability across the evaluated datasets.

Regarding comparisons with larger models, our evaluation is not limited to the Qwen family. Table 2 also includes Gemini 3.7 Flash, GPT-5.6, and GPT-4o. Sight-OPSD 4B achieves 91.12% balanced accuracy, surpassing Qwen3.5-397B (89.66%) and GPT-4o (90.43%), while remaining below Gemini 3.7 Flash (93.59%) and GPT-5.6 (92.64%). We clarify that these are pair-level verification comparisons of the Inspector, distinct from the full room re-identification results in Table 1.

Regarding geometric signals, the two stages serve related but distinct roles. In Stage I, the EMA teacher receives a training-only D1 visualization of cross-image RANSAC inliers to guide Inspector distillation. In Stage II, the Router uses a frozen Geometry Verifier (SuperPoint + LightGlue + RANSAC), providing correspondence statistics, spatial coverage, and calibrated LLRs. Thus, the same verifier and feature definitions are used during Router training and inference, although the Stage-I teaching views are not identical to the online geometric features.

To ensure a fair comparison with AirRoom, we retain its LightGlue-based matching backend. Our framework can accommodate alternative verifiers by recalibrating geometric LLRs on training data and retraining the Router accordingly. We appreciate this suggestion and systematic evaluation with alternative backends will be an interesting direction for future work. We have clarified these distinctions in Appendix B.1 and C.4.

Thank you for the rebuttal and addressing my concerns. My rating remains 3 (Weak acceptance).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.