acceptodds
Under review as a conference paper at ICLR 2027

LIMBO: Dynamic Map Evaluation Under Ambiguous Evidence

Abstract

Dynamic object maps must decide whether a detection belongs to a moved object, a replacement, a new arrival, or an object that has disappeared. Benchmarks usually score these decisions against one annotated history, although the same observations may admit several physical histories. We introduce LIMBO, an evaluation protocol built around sensor-matched compatible histories. It distinguishes an observed realization from a persistent entity and credits only claims shared by every history admitted under a declared evidence contract. In 355 controlled ProcTHOR groups, HOTA changes its preferred identity decision between hidden relocation and replacement even though the contracted mapper input is unchanged; 149 groups are also byte-identical in RGB-D. LIMBO preserves their common location and leaves lineage unresolved. In released 3RScan data, 5.92% of persistent targets in the sealed test split and 5.77% in a separate validation cohort have multiple annotation-compatible owners. The primary category-blind utility analysis finds no universal policy winner, whereas a separate audit of two released mappers exposes a coverage–precision trade-off. In ten fresh held-out simulated scenes, an online lifecycle mapper obtains 60.0% relocation retention, removal F1, and addition F1 from anonymous oracle regions. Simple RGB-D connected components reduce these scores to 20.0%, 10.7%, and 10.0%, showing strong sensitivity to proposal quality. LIMBO is intended for settings where unsupported certainty is costly. Realized-world and probabilistic scores remain appropriate for other decision objectives.

Primary area:
applications to robotics, autonomy, planning
Keywords:
dynamic semantic mapping, object-level mapping, data association, evaluation metrics, uncertainty quantification, embodied AI, RGB-D perception, 3D scene understanding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2XPfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper presents different evaluations of dynamic object identification, location, and pose in indoor, and generated, scenes, using a new way to reason about sameness. One such problem is the case where an object has undergone a hidden relocation, and then observed again; should it get a new ID, i.e., it is seen as a new object, or reuse the id of an already seen object, i.e., it has moved. The paper shows that the established metric HOTA will increase when using new ID on new objects, and when using old ID on moved objects, and otherwise decrease the score. In some cases it is impossible by observations to know which one case it is, and therefore using a metric susceptible to this in a benchmark may misrepresent the method under study. LIMBO determines how much information is available and what a method may know about the environment, which could be used to evaluate methods in a more fair way.

Also, using the grammar and the compatible groups, two new metrics are defined that are used together in an evidential F1 score. The results show that a quite small part of 3RScan contain ambiguities.

Critical strengths

A way to determine how much information is actually available in a given moment seems necessary to better judge how much can be expected from a method in this moment. In this paper it is accomplished by formalizing what has happened with an object over time by a grammar to express evidence contracts.

An evaluation seems to be done on the rate of scenes/worlds that shows ambiguousness which can influence the well established metrics like HOTA.

Critical weaknesses

It is difficult from the Results section to understand how LIMBO best is used in helping to evaluate trackers, mappers, etc. There are some numbers presented which may indicate that ambiguous interpretation of the environment may happen in around 5% of the scenes. Does this number indicate that methods having similar scores cannot determine a “winner” because the methods could have been “lucky”/“unlucky” to various degrees in these ambiguous scenes?

The presentation of the paper could be improved. More examples would be useful to introduce the problem. Describing use cases of LIMBO early in the paper and then using these use cases in the results section would improve the reading of the paper. Some things are introduced without proper definitions like evaluation contracts (why contracts, what makes them contracts. A contract guarantees something), typed evaluation protocol (why typed, what does it mean, and why protocol, what does it mean here). finite claim language (what means finite claim here). typed map claims needs to be explained (what means typed here. What means claims here).

The evidence ladder needs to be described in a clearer way. An introductory paragraph in section 5 describing what can be expected from the experiments would be helpful.

Figure 3 is not referenced anywhere in the text.

Questions for authors

Following my "Critical Weaknesses" assessment, the main question that the authors should clarify in the paper is: how is LIMBO used in helping to evaluate trackers, mappers, etc.? Additionally, I recommend the authors to carefully look at the "Critical Weaknesses" and address them.

4 comments

We thank you for your positive assessment of our soundness (Rating: 3) and core motivation, and for your constructive feedback on improving the presentation. In the uploaded revision, we have restructured the text, added upfront definitions, and clarified the practical workflows to address Q1-Q4 from your review of the original submission.

Q1 (Main Question & W1): How is LIMBO used in helping to evaluate trackers and mappers, and what does the ~5% ambiguity rate imply for determining a "winner"? We reorganized the opening of Section 5 (Page 6) around three concrete workflows showing how practitioners use LIMBO:

  • Eliminating Reference-Selection Luck (Secs. 5.1-5.2, App. L.4): As you noted, when observations admit multiple physical explanations (hidden relocation vs. identical replacement), single-reference metrics like HOTA and IDF1 reward whichever method is "lucky" enough to match the annotator's chosen trajectory. Across 223 held-out groups, varying the compatible reference history flips the preferred winner in 29 of 120 method pairs under HOTA and 32 of 120 pairs under IDF1 (with reuse-minus-replace HOTA swinging across [-0.159, +0.159] for fixed inputs and predictions; Table 44). LIMBO removes this luck factor by scoring claims against the intersection of all compatible histories.
  • Why Ambiguity Pervades Natural Scenes & Guides Decisions (revised Secs. 5.3-5.4): In revised Sec. 5.3, we clarify that while 5.92% (test) and 5.77% (val) of individual objects in 3RScan have multiple annotated owners, they compound across objects within a room, causing 37.2% of test scan pairs (122/328) and 43.6% of validation pairs (48/110) to admit multiple distinct global maximum matchings (Table 23)—and 88.17% of targets face multiple 3D geometric candidates (Table 24). On these scenes (Sec. 5.4, Table 3), LIMBO's category-gated global-certain reporting reduces worst-case decision cost by 36% (0.160 vs. 0.250) and cuts spatial map regret by 54.5% (0.136 m vs. 0.299 m).
  • Plug-and-Play Mapper Filtering (revised Sec. 5.5, Table 4): Wrapping existing mappers in LIMBO's evidence-scoped reporting (GLASSES + LIMBO Scope, Table 4) strips out 100% of unsupported lineage claims (31 to 0) while preserving 100% of matched 3D object locations (237/237) (see also Table 9 across 16 tracker configurations).

Q2 (W2): Clear Upfront Definitions of Core Terminology In revised Sec. 1 (Page 1) and Sec. 3.1 (Pages 3-4), we added bolded definitions for all four terms that lacked clear introductions in the original submission:

  • Typed map claims (Pages 1 & 3, Eq. 1): Formal assertions categorized into spatial realization predicates (AT(r,p), ABSENT_AT(r,p)—where an observed footprint is located or cleared now) and persistent entity lifecycle predicates (SAME_LINEAGE(r,e), BORN(e), REMOVED(e)—which physical lineage an observation belongs to and whether an entity entered or left the map).
  • Typed evaluation protocol (Page 1): A procedure compiling raw mapper outputs into these distinct claim types so valid spatial localization is credited independently of unresolved identity.
  • Evaluation contract (E, G) (Pages 1 & 4, Eq. 2): The formal specification pairing public sensor observations E with the finite grammar G of admitted physical mechanisms. It is a contract because it guarantees that any supported claim holds in the intersection ΓG(E)=⋂w∈WG(E)T(w)\Gamma_G(E) = \bigcap_{w \in \mathcal{W}_G(E)} T(w) across every compatible history.
  • Finite claim language CgC_g (Page 4): The finite set of candidate ground propositions obtained by instantiating Eq. (1) over the finite realizations, entities, and quantized poses of group g.

Q3 (W3): Clearer Evidence Ladder and Section 5 Introductory Paragraph

  • Evidence Ladder (revised Sec. 3.3, Pages 4-5): Rewritten with bolded level headers (Endpoint geometry, Endpoint plus free space, Passive continuous transport, Own action, Explicit import/export) linked directly to Table 6.
  • Section 5 Roadmap (Page 6): Added an opening roadmap paragraph at the start of Sec. 5 and streamlined all subsection headings (Secs. 5.1-5.6).

Q4 (W4): Missing Reference to Figure 3 Fixed in the revision: Figure 3 is now explicitly referenced and discussed in Sec. 5.4 (Line 418) and Sec. 6 (Line 517).

Thank you again for your constructive feedback. We would like to kindly follow up on our response and the revised manuscript.

Regarding your main concern about LIMBO's practical use, we have clarified three concrete evaluation workflows in Section 5. In particular, our new experiments show that reference selection can reverse method rankings in 29 of 120 comparisons under HOTA, illustrating why accounting for ambiguous histories matters in practice.

We have also substantially revised the presentation, including clearer definitions, additional explanations of practical use cases, and a reorganized experimental section to improve readability.

We hope these revisions help address your concerns. We would greatly appreciate any further feedback, especially whether our clarifications have resolved your main questions.

Thank you for your time and consideration.

Thanks to the authors for the revision of the document and consideration of the feedback.

It is still unclear if the possibility of metric flips (e.g., HOTA) will have a chance of changing the order of evaluated methods and if evaluating under this new way will affect the order. E.g., is there a chance that a list of state of the art methods is "wrong" and an evaluation under LIMBO changes the order?

To improve the readability it would be good to visualize how the different terms relate to each other and how they are used in the method, e.g., by extending Figure 1 or by introducing a new figure. Such a visualization could relate the terms via examples.

A new careful read through shows that line 298 says “We also evaluate the authors’ released GLASSES and …”, which could be interpreted as giving away the authors of the paper, but perhaps the sentence is misunderstood by the reviewer.

We sincerely thank you for re-reading our revised manuscript so carefully and for your constructive follow-up questions! Below we address all three points directly.

1. Can HOTA/accuracy be "wrong" and does evaluating under LIMBO actually change the ranking order of SOTA methods? Yes—evaluating under LIMBO directly reverses the ranking order of state-of-the-art mechanisms and released systems in both benchmarks:

  • Example A — SOTA Mechanism Ranking Reversal (Table 9, Page 16 & Table 13, Page 18): Consider comparing the SOTA active/missing object library mechanism (GLASSES, Bogenberger et al., 2026) against Track-continuity and Hard replace on held-out ProcTHOR groups:
    • Under conventional HOTA (wrongly favoring overreach): Hard replace (0.977) and Track-continuity (0.977) outrank GLASSES (0.864). Why? Because single-reference HOTA rewards guessing new IDs on replacement histories, ignoring that Hard replace commits 38.1% unsupported history errors.
    • Under LIMBO Evidential F1 (order reversed): The ranking flips completely. GLASSES (0.895, rising to 0.974 with LIMBO Scope at 0.0% unsupported error) decisively outranks both Track-continuity (0.811) and Hard replace (0.603), because LIMBO credits GLASSES for evidence-supported free-space clearing (+0.809 F1 gain on ray contracts; Table 13) and penalizes unsupported identity guessing.
  • Example B — Released 3RScan System Ranking Reversal (Table 4, Page 9 & Table 45, Page 33): Comparing the released ConceptGraphs (Gu et al., 2024) and GLASSES RGB-D pipelines across 2,789 targets:
    • Under standard association accuracy: ConceptGraphs ranks 1st (100.0%) and GLASSES ranks 2nd (78.0%–81.6%), because ConceptGraphs emits only 3 ultra-conservative stationary matches (0.1% recall).
    • Under LIMBO Evidential F1 (order reversed): GLASSES ranks 1st (0.082, and 0.083 with GLASSES + LIMBO Scope, retaining all 237 locations with 0 unsupported claims) while ConceptGraphs drops to last (0.001).

2. Visualizing How the Core Terms Relate via the Figure 1 Chair Example Thank you for this great suggestion! We have extended Figure 1 (and its caption walkthrough) to explicitly map each core term onto the concrete chair example:

  1. Evaluation Contract (E,G)(E, G) [Blue box, Fig. 1]: Pairs the public sensor input EE (calibrated depth C, mask D, pose, opaque handle m for the chair) with the admitted physical grammar GG (which allows both hidden relocation and twin replacement).
  2. Compatibility Group WG(E)\mathcal{W}_G(E) [Green & Orange boxes, Fig. 1]: All physical worlds matching EE under GG—here, World 1 (same chair m moved) and World 2 (chair m removed, identical chair other born at destination).
  3. Finite Claim Language Cg\mathcal{C}_g [Predicates inside boxes]: The finite set of candidate ground atoms instantiated for this chair: {AT(dest), SAME_LINEAGE(m), SAME_LINEAGE(other), REMOVED(m), BORN(other)}.
  4. Typed Evaluation Protocol & Supported Typed Map Claims ΓG(E)\Gamma_G(E) [Yellow "Intersection" box, Fig. 1]: Compiles mapper outputs into spatial realization claims (AT(dest)) vs. entity lifecycle claims (SAME_LINEAGE, BORN, REMOVED), and intersects World 1 and World 2 truths. Only AT(dest) survives in the intersection ΓG(E)\Gamma_G(E) (guaranteed true in every compatible history), while lineage/birth/removal remain unresolved (?).

3. Clarification on Line 298 ("the authors' released GLASSES...") Thank you for catching this phrasing! We are NOT the authors of GLASSES or ConceptGraphs—by "the authors' released GLASSES and ConceptGraphs...", we meant "the official open-source implementations publicly released by the respective authors of those cited papers (Bogenberger et al., 2026; Gu et al., 2024)." To prevent any ambiguity regarding double-blind review, we have updated Line 298 to read: "We also evaluate the official publicly released GLASSES and ConceptGraphs RGB-D mapping components ..."

Reviewer_TU3tOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on object identity uncertainty in dynamic map evaluation and proposes LIMBO, a framework that considers different possible object histories when evaluating object locations and identities. The authors test the framework on both simulated and real-world data. Overall, the paper addresses an interesting problem, but there are still some concerns about its dependence on predefined histories, practical benefits, and scalability.

Critical strengths

S1. The paper studies an interesting problem in dynamic map evaluation. It points out some limitations of existing metrics when object identities are uncertain and proposes a clear framework to address this issue.

S2. The experiments cover simulated data, real-world data, and existing mapping systems. These different settings help show how the proposed framework works.

Critical weaknesses

W1.In Section 5.2, Soundness drops from 100% to 81.1% and 87.8% when additional possible histories are included. This suggests that the results depend quite heavily on the predefined Grammar. It would be helpful to discuss how complete the Grammar is and whether adding more histories could change the ranking of different methods.

W2. In Section 5.5, LIMBO has a higher decision cost (0.268–0.292) than the always-unresolved baseline (0.250). It is not clear what practical benefits LIMBO offers in this case. Also, ConceptGraphs has only three evaluable associations in Section 5.6, which seems too few for a meaningful comparison between systems.

W3. LIMBO checks whether a claim holds across all compatible histories. However, the number of possible histories could grow quickly as more objects and longer sequences are considered. Does the method need to enumerate all possible histories? How would it handle larger scenes with many moving objects?

Questions for authors

W1-W3

3 comments

We thank you for highlighting the clarity of our framework and the breadth of our simulated, real-world, and released-system evaluations. In the uploaded revision, we added new ranking audits, frozen-protocol validation results, and polynomial-time complexity theorems to directly answer Q1-Q3 (W1-W3) from the original submission.

Q1 (W1): Grammar Dependence, Soundness Shifts (81.1% / 87.8%), Completeness, and Method Rankings

  • Why Soundness changed in original Sec. 5.2 (revised Sec. 5.2, App. K.5): Soundness shifted from 100% to 81.1% and 87.8% when we intentionally extended the reference grammar to admit unobserved mid-transport swaps (appearance-compatible replacement) and delayed observation. We clarified in revised Sec. 5.2 that this is an assumption audit: concluding from continuous transport that an object kept its identity requires assuming no identical twin was swapped in unobserved. Single-reference benchmarks bake this prior silently into one label; LIMBO measures the exact share of lineage claims (18.9% and 12.2%; Table 33) relying on it.
  • Effect on method rankings (new App. L.1, Tables 37-38): To answer your question on rankings, we added a full ranking audit across all 16 frozen configurations (120 pairs) on the 325 eligible groups. Admitting delayed observations and replacements shifts 26 pairwise orderings and creates 5 ties (Kendall's tau = 0.463 [0.384, 0.562], Spearman rho = 0.605), as aggressive reuse trackers lose support when swaps are admitted while conservative/scoped policies gain rank (Table 38).
  • Grammar completeness & 100%-sound spatial fallback (new App. L.6, Table 47): Across all leave-one-family-out grammar omissions on native endpoint groups across three disjoint house cohorts, LIMBO's spatial claims (AT, ABSENT_AT) remain 100% sound and 100% complete with zero unsupported assertions (Table 47). Spatial evaluation is completely invariant to lifecycle-grammar misspecification.

Q2 (W2): Practical Decision Benefits (original Sec. 5.5 -> revised Sec. 5.4) and Released Systems (original Sec. 5.6 -> revised Sec. 5.5)

  • 36% cost reduction under category gating (revised Sec. 5.4, Table 3, App. L.2): The 0.268-0.292 cost in the original Sec. 5.5 occurred only in the category-blind ablation, where semantic labels are stripped and raw XYZ geometry matches across all object classes (inflating candidate edges by 2.7x from 15,225 to 41,698 and dropping certain recall from 63.22% to 10.79%; Table 26). Even there, LIMBO achieves lower spatial map regret than abstention (0.259 m vs. 0.299 m). In revised Sec. 5.4 (Table 3), evaluating the standard category-gated contract on the official 47-environment validation cohort (110 pairs, 2,789 targets) shows that LIMBO reduces worst-case decision cost by 36% vs. always-unresolved (0.160 vs. 0.250, CI [-0.139, -0.026]) and cuts spatial map regret by 54.5% (0.136 m vs. 0.299 m) while resolving 65.6% of owner queries at zero query budget.
  • Clarifying ConceptGraphs & GLASSES (revised Sec. 5.5, Table 4): We revised Sec. 5.5 and Table 4 to clarify that this is not a leaderboard contest between ConceptGraphs and GLASSES, but a demonstration of two diagnostics: (1) Exposing recall collapse: ConceptGraphs' high-threshold static pipeline emits only 3 stationary-correct associations (100% precision but 0.1% recall), which LIMBO's coverage-aware Evidential F1 immediately diagnoses (0.001 vs. 0.082). (2) Overreach filtering: Adding GLASSES + LIMBO Scope to Table 4 shows that LIMBO eliminates 100% of unsupported GLASSES lineage claims (31 to 0) while retaining 100% of matched object locations (237/237).

Q3 (W3): Scalability—Does LIMBO Need to Enumerate All Possible Histories? (new Sec. 5.6, App. L.3)

  • No history enumeration: LIMBO never enumerates exponential global histories. In new Sec. 5.6 and App. L.3, we prove two exact polynomial reductions: (1) Matching + SCC Kernel: Computing one Hopcroft-Karp maximum matching followed by SCC reachability identifies all possible and certain edges in O(mn+n+m)O(m\sqrt{n} + n + m) time. (2) Linear Factorization (Eq. 8): Independent object histories factor linearly as ⋂w⋃iTi(wi)=⋃i⋂wiTi(wi)\bigcap_w \bigcup_i T_i(w_i) = \bigcup_i \bigcap_{w_i} T_i(w_i), avoiding 3N3^N Cartesian products.
  • 58-185x speedup (Table 43, Fig. 5): Across all 876 real 3RScan graphs, our exact kernel achieves a 58-185x speedup (scoring 328 test pairs in 0.21 s vs. 37.99 s, and 110 val pairs in 0.07 s vs. 12.64 s). For 1,024 moving objects (310243^{1024} combinations), factorized intersection takes 2.44 ms and SCC matching takes <55 ms (Fig. 5).

Thank you very much for authors response. I have raised my score.

Thank you very much for reconsidering our work and updating your assessment. We sincerely appreciate your time and consideration.

We noticed that the rating currently displayed is still 2, which is the same as the initial score. We just wanted to kindly check whether the updated rating has been successfully saved.

Thank you again for your support and feedback.

Reviewer_Big7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LIMBO: a useful way to evaluate what a robot’s map can legitimately claim from ambiguous observations. For example, images might establish that a mug is now on another table without revealing whether the original mug moved or was replaced. LIMBO credits claims shared by every history in a specified compatible set, while allowing identity to remain unresolved. Its conceptual case is stronger than its current evidence of practical benefit.

Critical strengths
  • Identifies an important evaluation distinction.
  • Strong controlled demonstrations.
  • Separates different kinds of claims.
  • Transparent reporting.
Critical weaknesses
  • Correctness depends heavily on the chosen history set.
  • Practical benefits remain unproven.
  • Real-system validation is limited.
  • Some central evidence is exploratory.
Questions for authors
  • How should users choose and validate the compatible-history grammar?
  • When does LIMBO lead to better downstream decisions?
  • How much real-world ambiguity is genuinely unavoidable?
  • Can the protocol evaluate sustained identity consistency?
  • Can the practical results be confirmed under a frozen protocol?
2 comments

We thank you for recognizing our core evaluation distinction, strong controlled demonstrations, and transparent reporting. In the uploaded revision, we have addressed Q1-Q5 and W1-W4 with new frozen-protocol validation results, grammar-omission guarantees, and exact graph-scoring benchmarks.

Q1 (and W1): How should users choose and validate the compatible-history grammar? The grammar G specifies admitted physical mechanisms (stationary, hidden relocation, replacement, birth, removal) paired with sensor interface E (Secs. 3.1, 6). In the revision, we added two formal diagnostics to validate G (revised Sec. 5.2, new App. L.1 & L.6):

  • 100%-Sound Spatial Fallback Under Omissions (new App. L.6, Table 47): Across all leave-one-family-out grammar omissions on native endpoint groups across disjoint house splits, LIMBO's spatial fallback (AT, ABSENT_AT) maintains 100% soundness, 100% completeness, 1.000 F1, and 0.000 overreach. Valid 3D object localization is completely immune to lifecycle-grammar misspecification.
  • Prior-Sensitivity Auditing (new App. L.1, Tables 37-38): Extending G to admit unobserved swaps or replacements quantifies the exact fraction of lineage claims (18.9% and 12.2%) and method rankings (Kendall's tau = 0.463) relying on continuous-persistence priors.

Q2 (and W2): When does LIMBO lead to better downstream decisions?

  • 36% cost & 54.5% regret reduction: Under standard category gating at zero query budget on the 47-environment 3RScan validation cohort (revised Sec. 5.4, Table 3), LIMBO's global-certain reporting reduces worst-case decision cost by 36% vs. abstention (0.250 to 0.160, paired diff. -0.090, 95% CI [-0.139, -0.026]), resolves 65.6% of owner queries, and cuts mean spatial map regret by 54.5% (0.299 m to 0.136 m).
  • Role of the observation contract: Comparing category-gated vs. category-blind interfaces (Table 26) shows that semantic gating removes 26,473 cross-class collisions (cutting candidate edges by 2.7x) and boosts certain true-owner recall nearly 6x (10.79% to 63.22% at 99.52% precision). With active query budgets (Fig. 3 right, Table 41), LIMBO's unresolved set guides active perception, driving map regret down to 0.004 m.

Q3 (and W3): How much real-world ambiguity is genuinely unavoidable, and how does it affect real systems?

  • Scene- & sensor-level prevalence (revised Sec. 5.3, Tables 23-24): While 5.92% (test) and 5.77% (val) of individual 3RScan targets have multiple annotated owners, they compound across rooms: 37.2% of test pairs (122/328) and 43.6% of val pairs (48/110) admit multiple global maximum matchings, and 88.17% of targets face multiple 3D geometric candidates.
  • Real-system validation (revised Sec. 5.5, Table 4, App. J): In revised Table 4, wrapping native GLASSES in GLASSES + LIMBO Scope on the 47-environment validation task eliminates 100% of unsupported lineage claims (31 to 0) while preserving 100% of matched object locations (237/237).

Q4: Can the protocol evaluate sustained identity consistency? Yes. Under 8-24 frame transport tubes and verified actions (Secs. 3.3, 5.2, Table 16), the compatible set contracts from 4 worlds to 1, crediting sustained SAME_LINEAGE continuity (F1: 0.667 to 1.000). We also evaluate TOMHT up to 4,096 global hypotheses (App. K.3, Tables 29-30). Moreover, our new Hopcroft-Karp + SCC graph algorithm and linear factorization (new Sec. 5.6, App. L.3, Eq. 8) accelerate exact scoring by 58-185x across 876 real graphs (Table 43) and resolve 1,024-object scenes in 2.44 ms (Fig. 5).

Q5 (and W4): Can the practical results be confirmed under a frozen protocol? Yes. While the original submission explored the 161-environment split, in revised Sec. 5.4 and new App. L.2 (Tables 3, 39-42) we locked the 14-feature XYZ scorer, calibration thresholds (95%/90%), and dev-selected rules on the train/dev splits and evaluated them untouched on the official 47-environment validation cohort (110 pairs, 2,789 targets), confirming the 36% worst-case cost reduction and 54.5% map-regret reduction.

Thank you again for your thoughtful review. We would like to kindly follow up on our response and the revised manuscript.

In particular, we have added grammar sensitivity analyses, validation under a frozen protocol showing a 36% reduction in worst-case decision cost, and real-system results demonstrating the elimination of unsupported lineage claims while preserving matched object locations.

We hope these additional results help address your concerns regarding grammar dependence, practical benefits, and real-system validation. We would appreciate any further feedback, particularly if any of your concerns remain unresolved.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.