acceptodds
Under review as a conference paper at ICLR 2027

PORTER: Edge–Cloud Residency for Persistent 3D Scene Graph Memory

Abstract

Recent task-driven and just-in-time 3D Scene Graph (3DSG) methods reduce per-task representations by constructing or activating only task-relevant information. Yet sparse per-task working sets do not bound onboard memory usage over a robot's lifetime: as tasks change, payloads accumulated for earlier tasks may become irrelevant to the current task but can be useful again in future tasks. Over repeated task switches and expanding environments, retaining such reusable payloads causes local memory to grow, whereas discarding them entirely can lead to costly repeated construction of the same payloads later. We introduce PORTER, which decouples persistence from residency: lightweight anchors remain in the limited memory of the edge robot while heavy object payloads migrate between the edge and the cloud. Relevance alone is insufficient for deciding residency because multiple relevant payloads may provide redundant information. We therefore decompose each task into functional requirements and introduce Irreplaceable Support Erasure (ISE), which measures the loss in requirement coverage caused by offloading. ISE discounts replaceable support and penalizes losses more strongly when the remaining coverage of a requirement is weak. PORTER constructs a budget-aware local working set by repeatedly offloading the payload with the smallest marginal ISE per byte. Experiments on JITOMA-Bench evaluate PORTER across four 3DSG builders. Under progressive compression, pooled relative mR@3 remains at 100% through 91% payload-byte offloading.

Primary area:
applications to robotics, autonomy, planning
Keywords:
3D scene graph, edge-cloud residency, persistent memory

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_neBZOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates retention of task-relevant information when using 3D scene graph representations as robot memories, for robots performing a sequence of tasks. The key idea is to retain a lightweight local memory of the information relevant to tasks, while offloading heavier representations to the cloud. The approach, called PORTER, quantified the loss of task information when offloading certain portions of the representation (payloads). The resulting metric is used in a submodular maximization problem to decide what to offload. The approach is tested in relevant benchmarks and shown to retain useful information while reducing the amount of information stored at the edge.

Critical strengths
  • The paper tackles an interesting problem and tension: how to retain task-relevant representations over long-deployments where the robot performs many different tasks.
  • The idea of looking at task support (essentially, understanding which tasks become infeasible if a payload is lost) instead of relevance is very interesting.
  • The presentation/writing is clear and easy to follow.
  • Formulating the problem as a submodular maximalization problem is quite elegant.
  • The experiments are quite comprehensive and apply PORTER to multiple scene graph memory construction approaches.
  • The paper also tested the approach in real-robot experiments.
Critical weaknesses
  • Equation (1) feels a bit ad hoc and is still related to task relevance: the paper would benefit from providing deeper foundations for this definition and the one in eq (5). Also, alpha and epsilon should be discussed right after eq (1). Moreover, after eq (1), it would be good to provide an example of how the metric can capture the fact that certain objects are redundant for a task (e.g. a sofa or chair for sitting), but other are simultaneously needed (e.g., sponge and dish soap for washing dishes).
  • If the weights \omega_r are all the same, why bother to include them in the definitions?
  • Table 1: strictly speaking, all objects (or at least their anchors) are retained by PORTER so the comparison in Table 1 seems confusing. It might be more relevant to compare memory reduction with and without PORTER. Same for Figure 4.
  • Clarity: Section 4.1: the description of Tier 2 and 3 tasks could be clearer in the experimental setup. In Section 3.2, please give an example of functional requirement to make the presentation clearer. The information rendered in Figure 2 might be better rendered as a 2D instead of a 3D plot.
  • Minor: considering that storage is cheap, one is left to wonder whether in practice one would send information out to the cloud or just store it to disk. The approach in the paper can easily handle both cases, and I feel that emphasizing the edge-cloud application might not be the strongest angle to take.
  • Minor: DAAAM (Gorlo et al, 2026) is used as a benchmark in the experiments but it is not discussed/mentioned in the related work section.
Questions for authors
  • Equation (1) feels a bit ad hoc and is still related to task relevance: the paper would benefit from providing deeper foundations for this definition and the one in eq (5). Also, alpha and epsilon should be discussed right after eq (1). Also: can you provide an example of how the metric can capture the fact that certain objects are redundant for a task (e.g. a sofa or chair for sitting), but other are simultaneously needed (e.g., sponge and dish soap for washing dishes)?
  • Table 1: strictly speaking, all objects (or at least their anchors) are retained by PORTER so the comparison in Table 1 seems a bit unfair or at least confusing. can you clarify why the number of objects was used as a key metric?
7 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/5)

We sincerely thank the reviewer for the thoughtful and constructive feedback. Your suggestions have been helpful in improving the clarity and theoretical motivation of our formulation. We respond to each point below.


W1: Equation (1) feels a bit ad hoc and is still related to task relevance: the paper would benefit from providing deeper foundations for this definition and the one in eq (5). Also, alpha and epsilon should be discussed right after eq (1). Moreover, after eq (1), it would be good to provide an example of how the metric can capture the fact that certain objects are redundant for a task (e.g., a sofa or chair for sitting), but other are simultaneously needed (e.g., sponge and dish soap for washing dishes).

A1: In response, we have revised the method section to better explain the intuition behind Eqs. (1) and (5), clarify the roles of the parameters, and introduce examples illustrating redundancy and complementarity.

(1) Anchor-based support (Eq. 1). Equation (1) estimates how strongly an individual object supports a task requirement based on its anchor embedding, yielding a full-payload support score. The corresponding anchor-only support is scaled by η\eta, reflecting the support retained after payload offloading. Since anchors remain locally available after offloading, this allows PORTER to estimate task support without first retrieving the full payload. Here, α\alpha filters weak semantic matches and rescales stronger ones, while ε\varepsilon caps individual support below 1 to prevent complete saturation in the subsequent Noisy-OR aggregation.

(2) Redundancy and complementarity. Following the reviewer's suggestions, for illustration, assuming η=0\eta=0, consider a task requiring a place to sit, with a sofa and a chair each providing a support score of 0.8 for the corresponding functional requirement. Following the Noisy-OR aggregation, their joint coverage is 1−(1−0.8)2=0.961-(1-0.8)^2=0.96. Offloading either object reduces coverage by only 0.16, since the other can still provide the required support. However, once only one object remains, offloading it causes a further loss of 0.8.

Resident payloadsCoverageLoss from previous state
Sofa + Chair0.96—
Sofa or Chair0.800.16
Neither00.80

Consequently, the marginal task-support loss, and thus the resulting ISE increase, depends on whether alternative objects can still provide support for the same requirement, rather than simply on the individual support score of the offloaded object.

In contrast, washing dishes may require both a cleaning tool (sponge) and a cleaning agent (dish soap). Since coverage is computed separately for each functional requirement, additional sponges cannot compensate for the absence of dish soap. This allows ISE to account for both interchangeable objects and objects that serve different, complementary requirements.

(3) Log-survival loss (Eq. 5). ISE measures the relative task support lost after offloading compared with full residency. The support-survival ratio ρr(S)\rho_r(S) measures the fraction of the full-residency support that remains locally available for requirement rr. Minimizing its negative log-survival loss is equivalent to maximizing the weighted geometric mean of requirement-wise survival ratios:

∏r∈Rqρr(S)ωr\prod_{r\in\mathcal R_q}\rho_r(S)^{\omega_r}

This formulation ensures that a requirement with very little support remaining has a strong effect on the loss, rather than having its loss simply offset by high support for other requirements.

We have incorporated these explanations, parameter definitions, and illustrative examples into Secs. 3.2–3.3 of the revised manuscript.

AuthorsOpenReview ↗

Rebuttal by Authors (2/5)

W2: If the weights ωr\omega_r are all the same, why bother to include them in the definitions?

A2: We retain ωr\omega_r to make the formulation flexible to different priorities among functional requirements. For example, in a meal-preparation task, locating the main ingredient may be assigned a higher priority than locating a serving utensil, depending on the task objective.

In this work, however, we do not estimate task-dependent priorities and adopt uniform weights (ωr=1/M\omega_r=1/M) as a simplifying assumption. Thus, the weighted formulation provides flexibility for scenarios with non-uniform requirement priorities, while our current experiments use the uniform-weight setting. Exploring task-dependent requirement weighting is an interesting direction for future work.

AuthorsOpenReview ↗

Rebuttal by Authors (3/5)

W3: Table 1: strictly speaking, all objects (or at least their anchors) are retained by PORTER so the comparison in Table 1 seems confusing. It might be more relevant to compare memory reduction with and without PORTER. Same for Figure 4.

A3: We thank the reviewer for pointing out this ambiguity. We originally reported the number of locally resident payload objects to remain consistent with the object-level evaluation protocol of JITOMA-Bench. However, we agree that this metric requires clarification: the reported count refers to objects whose full payloads remain locally resident, not the total number of objects or anchors retained by PORTER. PORTER retains lightweight anchors and persistent IDs for all objects, while only a subset of their full payloads remains locally available.

(3.1) Anchor versus payload memory. To illustrate why resident-payload object count provides a useful coarse indication of memory usage, we measured the average payload and anchor sizes in the three ConceptGraphs scenes used in Table 1:

Scene# ObjectsMean payload/objectMean anchor/objectRatio
Apartment45238.68 MiB4.19 KiB9,453×
Office92422.54 MiB4.19 KiB5,509×
Cubicle22447.42 MiB4.19 KiB11,589×

As shown above, individual payloads are thousands of times larger than their corresponding anchors. Thus, although all object anchors remain locally available, the number of locally resident payloads provides a useful coarse indication of local object-state memory usage. We acknowledge that object count alone is not a direct measurement of memory consumption.

As an additional aggregate comparison, the trajectory-averaged resident-payload footprint decreases from 10.37–20.17 GiB in Vanilla ConceptGraphs to 0.64–1.02 GiB with PORTER, corresponding to a 92.4–96.2% reduction across the three scenes. These supplementary measurements directly demonstrate the reduction in resident payload memory.

We have clarified the meaning of the resident-payload count in Sec. 4.1 and the corresponding Table 1 and Figure 4 descriptions, and added the detailed anchor–payload memory analysis in Appendix A.3.5.

Reviewer_Q6LVOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper at hand presents PORTER, an algorithm for making decision of which parts of a 3D Scene Graph to offload from local memory in a task-dependent manner. For this, it separates 3D scene graph nodes into anchors and payloads. It then introduces a mechanism termed ISE (Irreplacable Support Erasure) which measures the support of each graph vertex to a set of task requirements. PORTER then reverse-greedily offloads payloads with the smallest task support (i.e., ISE per byte). The authors show in experiments that this approach works across different 3DSG creation frameworks and reduces the number of payloads that need to be stored locally.

Critical strengths

The authors importantly, and in contrast to many other publications, discuss and consider robot operation over the lifetime of a robot, instead of just about the horizon of a singular task. This is an important direction for the community to move towards as methods grow more mature. Further, the authors provide substantial experimentation by applying their method to four other baseline methods.

Critical weaknesses

The paper contains two critical weaknesses:

It is not clearly motivated and substantiated (with evidence) why the problem the authors are addressing is important. The paper differentiates between local and cloud memory, however it is not clear whether they mean working memory (RAM) or storage. The authors do not provide evidence for the raw amount of data they are addressing, making it hard to tell whether the problem they are addressing is a major problem for a large number of use cases or an issue that only affects certain special cases. Additionally, the usage of the cloud is not clearly motivated - local storage is, even for small mobile robots, cheap and readily available in the order of few TBs.

The second weakness is the approach itself. A decision on offloading is taken purely based on information that is not offloaded (AvA_v). ISE provides a measure only based on the anchor information, but not based on the payload, so whether the payload is actually needed for the task is unknown. The authors don't provide a reason for why the payload would be important and why importance can be determined only by anchors. Additionally, and more importantly even, the method is positioned as a plug-in method for any 3DSG-based system. This requires that both the scene-graph-creation method as well as a robot policy operating with the scene graph as an input can keep working without any changes. Critically however, PORTER can have an effect on task performance by altering the set of available observations for the robot policy. If positioned as a general system technology, this should not be the case as the decision of which objects are relevant needs to be taken by the policy or the perception (3DSG) method. An example here could be continued scene graph building: By offloading payloads, a 3DSG method will not be able to update its scene graph based on new observations since older observations are not available anymore and can only be made available in the context of a task.

Questions for authors

Besides the questions implied in "Critical Weaknesses", there are a few other questions that come to mind:

Why should the relevance of objects (and their payloads) for a task only be described by a set of vision-language-embeddings?

How is "relevance to a task" different from "what task support is lost if the node leaves the edge"?

Why should "reshaping the local working set" considered as a positive effect, as described in A.7? Should this not be the responsibility of the 3DSG method or robot policy, instead of the "residency manager"?

8 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/6)

We thank the reviewer for raising these important questions about the memory resource managed by PORTER. We have added a discussion in Sec. 4.7 to clarify these aspects.


W1: It is not clearly motivated and substantiated (with evidence) why the problem the authors are addressing is important. The paper differentiates between local and cloud memory, however it is not clear whether they mean working memory (RAM) or storage. The authors do not provide evidence for the raw amount of data they are addressing, making it hard to tell whether the problem they are addressing is a major problem for a large number of use cases or an issue that only affects certain special cases. Additionally, the usage of the cloud is not clearly motivated - local storage is, even for small mobile robots, cheap and readily available in the order of few TBs.

A1: We realize that our original manuscript did not sufficiently distinguish working memory from persistent storage or clarify the role of cloud storage in our design. We address these concerns from three aspects.

(1.1) Working memory versus persistent storage. In this work, local memory refers to the edge-side working memory (RAM) occupied by object payloads that are actively available to online 3DSG construction and task execution, rather than the total capacity of persistent local storage. PORTER does not aim to store all historical data in RAM; instead, it manages which object payloads remain in the active working set and which can be temporarily offloaded while retaining their lightweight anchors locally.

Even when a robot has sufficient local disk capacity to store the complete scene graph, payloads stored on disk must still be loaded into RAM before they can participate in online observation updates, object-level fusion, or task execution. Therefore, ample local storage does not eliminate the need to manage the active working set under a limited RAM budget.

(1.2) Quantifying the payload footprint. To provide concrete evidence of the amount of data being managed, we additionally measure the trajectory-averaged resident-payload footprint in the three Tier 2 ConceptGraphs scenes reported in Table 1. Specifically, at each frame, we sum the actual data sizes of all locally resident object payloads, and average this sum over the entire trajectory. The results are shown below.

SceneVanilla ConceptGraphsConceptGraphs + PORTERReduction
Cubicle10.372 GiB0.785 GiB92.4%
Apartment17.071 GiB0.643 GiB96.2%
Office20.166 GiB1.023 GiB94.9%

As shown above, Vanilla ConceptGraphs maintains an average resident-payload footprint of 10.4–20.2 GiB across these scenes, whereas PORTER reduces it to 0.64–1.02 GiB, corresponding to a 92.4–96.2% reduction. These measurements demonstrate that the object payloads managed by PORTER can constitute a substantial working-set footprint, rather than merely small metadata. While the actual memory pressure depends on the available RAM and deployment, these results show that multi-GiB payload footprints can arise in practical 3DSG scenarios. Importantly, these numbers measure resident object payloads only, and do not represent total device RAM consumption, as lightweight anchors and other runtime memory allocations are not included.

(1.3) Storage-tier-independent deployment. We agree that local SSDs provide an inexpensive and practical option for persistent data storage, and we do not claim that cloud storage is universally necessary or preferable to local storage. Our focus is task-conditioned residency management across storage tiers, rather than the choice of the backing medium itself.

The ISE objective evaluates task support relative to payload memory footprints, while the Push/Pull abstraction manages payload transfers between storage tiers. Neither is inherently tied to cloud storage. The same ISE-based residency principle can be applied between RAM and local SSD, or hierarchically in a multi-tier system where local SSD serves as an intermediate tier between RAM and cloud storage. Thus, even when a robot has ample local disk capacity, PORTER can manage which payloads remain in RAM and which reside in other storage tiers. In our implementation, we use cloud storage as the remote backing tier to instantiate an edge–cloud deployment. This is one representative configuration of PORTER, rather than a requirement of the proposed method.

We have clarified the distinction between working memory and persistent storage, as well as PORTER's generality across storage tiers, in Sec. 4.7 (Practical Deployment and Usage).

AuthorsOpenReview ↗

Rebuttal by Authors (2/6)

W2: The second weakness is the approach itself. A decision on offloading is taken purely based on information that is not offloaded (Av). ISE provides a measure only based on the anchor information, but not based on the payload, so whether the payload is actually needed for the task is unknown. The authors don't provide a reason for why the payload would be important and why importance can be determined only by anchors.

A2: The key distinction is that anchors identify which objects are likely to be relevant to a task, whereas payloads provide the detailed information needed to use these objects in downstream applications. This follows the common usage of 3DSG-based systems, which can be conceptually viewed as involving two stages.

(2.1) Semantic retrieval versus downstream execution. In the first stage, a robot identifies objects relevant to a task using semantic and spatial information in the scene graph. For example, given a natural-language instruction, it needs to retrieve objects satisfying the requested semantic or functional requirements. Recent scene-graph-based navigation methods, such as SG-Nav [1] and PSG-Nav [2], demonstrate how scene graph information can support target identification and navigation. This motivates our use of lightweight anchors for estimating object-level task relevance.

In the second stage, downstream applications may require detailed object information to execute the task. For example, point clouds provide geometric information for precise localization and point-cloud-based perception, while historical image crops preserve previous visual observations for appearance analysis and temporal reasoning. These detailed representations are stored in payloads. Thus, payloads are essential for task execution, even though their full contents are not necessarily needed to identify task-relevant objects.

(2.2) Why anchor-based residency decisions? Anchors retain object identities and lightweight semantic information, particularly vision-language embeddings. Given the functional requirements of a task, this information allows PORTER to estimate which objects are likely to provide task support without retrieving their full payloads. This is important because residency decisions must be made before deciding whether to retrieve a payload. Requiring full payload inspection would introduce additional retrieval and memory overhead, undermining the purpose of residency management. Accordingly, ISE measures the loss of object-level task support based on anchors, rather than attempting to estimate the utility of every individual component within a payload.

(2.3) Scope of the design. Importantly, we do not assume that anchors contain all information required by downstream tasks or can replace payloads. The two components serve complementary roles: anchors support task-relevant object retrieval, while payloads provide the detailed geometric, visual, and historical information required for subsequent execution. We acknowledge that highly fine-grained tasks whose requirements depend on details not captured by anchors (e.g., identifying an object by a small scratch) may require richer anchor representations or additional retrieval mechanisms. Such cases are beyond the scope of our current object-level residency formulation.

Overall, PORTER uses anchors to determine which objects should remain readily accessible, while preserving their full payloads for retrieval when detailed information is needed by downstream tasks.

[1] Yin, H., Xu, X., Wu, Z., Zhou, J., & Lu, J. (2024). Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation. Advances in neural information processing systems, 37, 5285-5307.

[2] Chen, R., Chang, Y., Tang, X., Chen, H., & Xie, S. (2026). Psg-nav: Probabilistic scene graph navigation via multiverse decision making. arXiv preprint arXiv:2606.01313.

AuthorsOpenReview ↗

Rebuttal by Authors (3/6)

W3: Additionally, and more importantly even, the method is positioned as a plug-in method for any 3DSG-based system. This requires that both the scene-graph-creation method as well as a robot policy operating with the scene graph as an input can keep working without any changes. Critically however, PORTER can have an effect on task performance by altering the set of available observations for the robot policy. If positioned as a general system technology, this should not be the case as the decision of which objects are relevant needs to be taken by the policy or the perception (3DSG) method. An example here could be continued scene graph building: By offloading payloads, a 3DSG method will not be able to update its scene graph based on new observations since older observations are not available anymore and can only be made available in the context of a task.

A3: We clarify that PORTER is a plug-in residency-management layer that controls where object payloads reside without replacing the native processing algorithms or interfaces of the upstream 3DSG builder and downstream policy. We further explain how continued mapping is handled under our default observation policy and provide a Drop-versus-Buffer comparison in Sec. 4.7 and Appendix A.9.

(3.1) Plug-in residency management. Under a bounded working-memory budget, when the resident payload footprint exceeds available RAM, a 3DSG system must decide which payloads remain locally resident and which are moved to another storage tier or discarded. PORTER addresses this problem through task-conditioned residency decisions rather than simply discarding historical objects. Importantly, task interpretation and functional-requirement specification remain external to PORTER: it uses these requirements to prioritize payload residency, rather than replacing the policy's target selection or the builder's perception and association decisions. Its plug-in nature means that the builder's native processing algorithms and the downstream policy interface remain intact, while PORTER independently manages payload residency. The local working set necessarily changes under this process, but this is precisely the resource-management function PORTER is designed to provide.

(3.2) Continued mapping and the Drop design. PORTER retains all lightweight anchors and persistent object IDs locally, so incoming observations can still be associated with existing objects using the upstream builder's native association procedure. A match to an offloaded object reuses its original ID without creating a duplicate node or triggering Pull. Under our default Drop design, the matched observation's payload update is discarded, and the cloud-resident payload remains frozen until a task-driven Pull restores it.

We deliberately adopt Drop to minimize modifications to existing 3DSG builders. It preserves their native association and fusion procedures without requiring additional observation-buffering or replay interfaces. In contrast, Buffer requires additional integration with the upstream update pipeline: matched observations must be intercepted, stored together with their object associations, and replayed for fusion after the corresponding payload is restored. Thus, although Buffer preserves more intermediate observations, it introduces additional storage and replay requirements. After Pull, both designs resume the builder's native payload update and fusion process.

(3.3) Drop versus Buffer. To examine the trade-off between observation preservation and integration complexity, we compare the default Drop policy with Buffer using ConceptGraphs on the three Tier 2 scenes reported in Table 1. Buffer temporarily retains observations associated with offloaded objects and replays them into the restored payload using the builder's native fusion procedure.

(3.4) Results. The following table compares the two policies. Each pair is ordered as Drop / Buffer.

SceneIoU (%)mR@1 (%)mR@3 (%)
Apartment17.6 / 17.733.3 / 33.355.6 / 55.6
Office19.2 / 19.440.5 / 40.545.2 / 45.2
Cubicle12.4 / 12.422.2 / 22.263.0 / 63.0

The results show that Buffer leaves mR@1 and mR@3 unchanged across all three scenes, with only marginal IoU changes (at most 0.2 percentage points). Thus, retaining and replaying observations during offloading provides no measurable improvement in grounding recall in this Tier 2 setting. These results support our choice of Drop as the default integration strategy: it avoids additional observation-storage and replay interfaces while maintaining the same evaluated grounding recall.

Reviewer_mjFhOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies edge–cloud residency for persistent 3D scene graph memory. PORTER keeps lightweight anchors on the robot and moves heavy object payloads between the edge and cloud. Its Irreplaceable Support Erasure (ISE) objective guides payload selection under a local memory budget. Experiments across four 3DSG builders and a real robot show reduced local residency while largely preserving mR@3.

Critical strengths
  1. Separating persistence from residency addresses a practical problem in long-term robotic memory.
  2. ISE accounts for substitute support and weak remaining requirement coverage, while the scheduler considers payload size.
Critical weaknesses
  1. The benefit over lightweight summaries with on-demand Pull remains unclear. Tier 3 evaluates only objects whose payloads remain local, although their vision–language embeddings persist in anchors. This protocol measures the capability of the resident payload set, but leaves a simpler alternative untested, i.e., retaining embeddings and predicted 3D boxes locally, then using Pull only when detailed object state is needed. Comparing this baseline under the same total local-memory budget and network conditions, with grounding accuracy and Pull overhead reported, would clarify the benefit of PORTER's task-conditioned residency.

  2. The high-compression gains are not clearly linked to the online operating points. Table 2 reports Overall nAUC of 98.30% for PORTER, 97.95% for w/o Log-Information Loss, and 98.27% for w/o Dynamic Re-ranking. The tail gains are clear, and Sec. 4.5 explicitly treats this region as a stress test rather than a nominal operating point. However, the payload-byte budgets used in Tier 2 and the real-robot experiments are not reported. Showing how these budgets relate to the full payload size, and comparing the variants at those operating points, would establish when log-survival and dynamic re-ranking provide practical benefits.

  3. Data association before Pull needs a clearer explanation. Sec. 3.5 and Appendix A.3.1 explain that cloud-resident payloads remain frozen and resume native updates after Pull. However, it is unclear how new observations of these objects are handled before Pull. Please explain how an incoming observation is matched to a persistent anchor, whether this triggers Pull, and how duplicate local objects are avoided. If a new local representation is created, how is it reconciled with the restored payload? This question concerns local data association during continued mapping, rather than cloud-side synchronization.

Questions for authors

Please see weaknesses. I'm willing to raise my score if my concerns are addressed.

5 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/3)

We thank the reviewer for the constructive and thoughtful feedback. We respond to each point below:


W1: The benefit over lightweight summaries with on-demand Pull remains unclear. Tier 3 evaluates only objects whose payloads remain local, although their vision–language embeddings persist in anchors. This protocol measures the capability of the resident payload set, but leaves a simpler alternative untested, i.e., retaining embeddings and predicted 3D boxes locally, then using Pull only when detailed object state is needed. Comparing this baseline under the same total local-memory budget and network conditions, with grounding accuracy and Pull overhead reported, would clarify the benefit of PORTER's task-conditioned residency.

A1: We agree that directly comparing PORTER against a lightweight-summary-based, on-demand retrieval strategy is important for establishing the benefit of task-conditioned residency. We agree that Tier 3 alone does not fully establish the benefit of PORTER over the simpler alternative described by the reviewer. In particular, evaluating only the payloads that remain resident does not directly compare task-conditioned residency against retaining lightweight anchors and retrieving detailed payloads on demand. We therefore implemented an On-demand LRU baseline following this alternative and added a direct comparison in Sec. 4.7. We describe the baseline, ensure matched memory and network conditions, and report both grounding accuracy and retrieval overhead below.

(1.1) Baseline comparison. Both methods retain lightweight local anchors. On-demand LRU retrieves missing candidate payloads only when a query requires them and manages the local cache based on access recency. PORTER additionally performs task-conditioned prefetching at task switches and makes eviction decisions based on marginal ISE loss per byte.

FeatureOn-demand LRUPORTER
Persistent lightweight anchors and global retrieval✓✓
Query-time Pull and cache reuse✓✓
Task-conditioned proactive Pull✗✓
Eviction criterionLRUMarginal ISE loss per byte

(1.2) Experimental setup. We evaluate both methods using ConceptGraphs on the three Tier 2 scenes, with identical precomputed observations, task requirements, and network conditions. Importantly, we match the total managed-memory budgets, accounting for anchors, resident payloads, observation buffers, and temporary-operation reserves. Tight, Medium, and Loose correspond to 10%, 25%, and 50% of the original peak managed-memory footprint, respectively.

(1.3) Results. The following table reports grounding accuracy and retrieval overhead, averaged over the three scenes. Each pair is ordered as On-demand LRU / PORTER.

BudgetmRecall@1 (%)mRecall@3 (%)Query-triggered Pulls/queryMean query wait (s)
Tight (10%)30.5 / 38.147.6 / 55.23.09 / 3.261.04 / 1.29
Medium (25%)30.5 / 38.147.6 / 55.22.69 / 3.200.83 / 1.17
Loose (50%)29.5 / 36.248.6 / 54.32.37 / 3.120.81 / 1.12

PORTER consistently achieves higher grounding accuracy across all three memory budgets, improving mRecall@1 by 6.7–7.6 percentage points and mRecall@3 by 5.7–7.6 percentage points. This improvement comes with increased retrieval overhead, as reflected by more query-triggered Pulls and longer query waiting times.

(1.4) Why does task-conditioned residency improve grounding? Beyond making task-relevant payloads available proactively, prefetching also allows these objects to receive and fuse new observations throughout task execution. In contrast, under On-demand LRU, offloaded payloads remain frozen until query-time Pull, so observations collected during their offloading period cannot be incorporated before grounding. Consequently, the restored object states may be less up-to-date when the query is performed. This provides an additional explanation for the grounding accuracy gap between the two methods.

We have incorporated the baseline, experimental setup, results, and discussion into Sec. 4.7 and Fig. 5.

AuthorsOpenReview ↗

Rebuttal by Authors (2/3)

We thank the reviewer for highlighting the importance of connecting our component ablations with practical operating points. We clarify the budget used in our main experiments and provide an additional Tier 2 ablation under the same setting.


W2: The high-compression gains are not clearly linked to the online operating points. Table 2 reports Overall nAUC of 98.30% for PORTER, 97.95% for w/o Log-Information Loss, and 98.27% for w/o Dynamic Re-ranking. The tail gains are clear, and Sec. 4.5 explicitly treats this region as a stress test rather than a nominal operating point. However, the payload-byte budgets used in Tier 2 and the real-robot experiments are not reported. Showing how these budgets relate to the full payload size, and comparing the variants at those operating points, would establish when log-survival and dynamic re-ranking provide practical benefits.

A2: Although 99% represents an aggressive compression setting, it is also the operating point adopted in both our Tier 2 benchmark experiments and real-robot deployment. We have further conducted an additional Tier 2 component ablation under this same setting to directly examine the practical contributions of individual components.

(2.1) Operating-point clarification. In both Tier 2 and real-robot experiments, the target resident-payload budget is set to 1% of the baseline's peak aggregate payload-byte footprint, corresponding to a nominal 99% reduction relative to that peak. Specifically, the reference footprint is the maximum, over the baseline trajectory, of the total bytes of locally resident object payloads. The budget is builder- and scene-dependent rather than a fixed byte value shared across settings. This is the target at each residency update; subsequent online mapping can accumulate new resident payloads until the next update. Lightweight anchors are retained and excluded from this ratio. We have clarified this budget definition in Sec. 4.2 and specified the same operating point for the real-robot deployment in Sec. 4.4 (Fig. 4).

(2.2) Tier 3 ablation at the 99% operating point. We note that Table 2 in Sec. 4.5 already includes the 99% payload-byte offloading checkpoint. At this point, PORTER retains 59.02% of the Keep-All mR@3, compared with 36.07% without Noisy-OR, 45.90% without Byte Normalization, 50.82% without Log-Information Loss, and 54.10% without Dynamic Re-ranking. Thus, the component contributions are already observable at the same nominal offloading ratio used in our online experiments.

(2.3) Additional Tier 2 ablation. To further examine these contributions under the actual Tier 2 operating conditions, we conducted an additional component ablation using ConceptGraphs on all three Tier 2 scenes, with the same 99% offloading target. The following table reports absolute grounding recall. Each pair is ordered as mR@1 / mR@3.

MethodApartmentOfficeCubicle
PORTER33.3 / 55.640.5 / 45.222.2 / 63.0
w/o Noisy-OR27.8 / 47.231.0 / 40.514.8 / 48.2
w/o Byte Normalization27.8 / 50.035.7 / 40.518.5 / 55.6
w/o Log-Information Loss30.6 / 50.035.7 / 40.522.2 / 59.3
w/o Dynamic Re-ranking33.3 / 52.838.1 / 42.922.2 / 59.3

These results are reported in Appendix A.8 (Table 13), and we have added a pointer to this complementary evaluation in Sec. 4.5.

(2.4) Discussion. PORTER matches or outperforms all ablated variants across the three scenes. Averaged over the scenes, removing Noisy-OR reduces mR@1 and mR@3 by 7.5 and 9.3 percentage points, respectively; removing Byte Normalization reduces them by 4.7 and 5.9 points; removing Log-Information Loss reduces them by 2.5 and 4.7 points; and removing Dynamic Re-ranking reduces them by 0.8 and 2.9 points.

The differences are smaller than those observed in Tier 3, which is expected because Tier 2 primarily involves single-target queries and is less discriminative of redundancy and multi-requirement support. Nevertheless, the additional results show that all four components contribute positively even at the practical operating point. In particular, Log-Information Loss and Dynamic Re-ranking provide measurable benefits beyond the overall compression-curve performance reported in Table 2.

We have incorporated the operating-point clarification into Secs. 4.2 and 4.4, and the additional Tier 2 ablation and discussion into Appendix A.8, with a corresponding reference in Sec. 4.5.

AuthorsOpenReview ↗

Rebuttal by Authors (3/3)

W3: Data association before Pull needs a clearer explanation. Sec. 3.5 and Appendix A.3.1 explain that cloud-resident payloads remain frozen and resume native updates after Pull. However, it is unclear how new observations of these objects are handled before Pull. Please explain how an incoming observation is matched to a persistent anchor, whether this triggers Pull, and how duplicate local objects are avoided. If a new local representation is created, how is it reconciled with the restored payload? This question concerns local data association during continued mapping, rather than cloud-side synchronization.

A3: We thank the reviewer for pointing out that our original explanation did not sufficiently clarify local data association during continued mapping. We have clarified the association and observation-handling mechanism in Sec. 4.7, and added a detailed Drop-versus-Buffer comparison in Appendix A.9. Our key design choice is to preserve the upstream 3DSG builder's native association procedure while managing payload residency separately.

(3.1) Data association without Pull. PORTER retains lightweight anchors and persistent object IDs locally, including for objects whose payloads have been offloaded. Incoming observations are matched against these anchors using the upstream builder's native association procedure. If an observation matches an offloaded object, it reuses the original object ID without creating a duplicate node. This association does not trigger Pull; Pull is initiated only by the task-driven residency policy when the full payload is needed. Consequently, no separate local object is created, and the restored payload continues under the same persistent object ID without requiring a separate reconciliation process.

(3.2) Default Drop policy. Under our default Drop policy, the observation is associated with the original object, but its payload-level update is discarded, leaving the cloud-resident payload frozen until a task-driven Pull restores it. We adopt this design to minimize modifications to existing 3DSG builders: it preserves their native association logic without requiring additional observation-buffering and replay interfaces. After Pull, the restored payload resumes the builder's native update and fusion process. The trade-off is that observations received during offloading are not incorporated into the payload.

(3.3) Drop versus Buffer. To examine this trade-off, we additionally compare Drop with a Buffer variant using ConceptGraphs on the three Tier 2 scenes. Unlike Drop, Buffer temporarily retains observations associated with offloaded objects and replays them into the restored payload through the builder's native fusion procedure after Pull. While this preserves observations received during offloading, it requires additional observation storage and replay interfaces.

(3.4) Results. The following table compares the grounding performance of the two policies. Each pair is ordered as Drop / Buffer.

SceneIoU (%)mR@1 (%)mR@3 (%)
Apartment17.6 / 17.733.3 / 33.355.6 / 55.6
Office19.2 / 19.440.5 / 40.545.2 / 45.2
Cubicle12.4 / 12.422.2 / 22.263.0 / 63.0

Buffer leaves mR@1 and mR@3 unchanged across all three scenes, with only marginal IoU changes (at most 0.2 percentage points). Thus, retaining and replaying observations during offloading provides no measurable improvement in grounding recall in this Tier 2 setting. These results support our default Drop policy as a lightweight integration strategy that avoids additional observation-storage and replay interfaces while maintaining the same evaluated grounding recall.

We have incorporated the association explanation and the main experimental findings into Sec. 4.7, with the detailed experimental setup and results provided in Appendix A.9 (Table 14).

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

Dear Reviewers and Area Chair,

We sincerely thank all reviewers for their thoughtful and constructive feedback. We have provided detailed point-by-point responses and revised the manuscript accordingly. Below is a brief summary.

Summary of revisions:

  • On-demand baseline (mjFh, W1). Added an On-demand LRU comparison under matched memory and network conditions. PORTER improves mRecall@1 by 6.7–7.6 points and mRecall@3 by 5.7–7.6 points, with increased retrieval overhead. (Sec. 4.7, Fig. 5)

  • Practical operating-point ablation (mjFh, W2). Clarified the 99% nominal payload-offloading target used in Tier 2 and real-robot experiments, and added Tier 2 component ablations at this operating point. (Secs. 4.2, 4.4–4.5, Appendix A.8)

  • Continued mapping (mjFh, W3; Q6LV, W3). Clarified persistent-anchor association and the default Drop policy. Added a Drop-versus-Buffer comparison, showing unchanged grounding recall across three Tier 2 scenes. (Sec. 4.7, Appendix A.9)

  • ISE formulation and design rationale (Q6LV, W2/W4–W6; neBZ, W1–W2). Expanded the explanation of anchor-based task support, redundancy, complementarity, and log-survival loss. Further clarified the distinctions between task relevance, marginal support loss, and residency management. (Secs. 3.2–3.3; detailed responses)

  • Memory accounting and deployment (Q6LV, W1; neBZ, W3/W5). Clarified resident-payload object counts, added per-object anchor–payload memory analysis, and explained PORTER's applicability beyond edge–cloud storage. Additional aggregate payload-footprint measurements are provided in our responses. (Secs. 4.1, 4.7, Appendix A.3.5)

  • Presentation and related work (neBZ, W4–W5). Clarified Tier 2/3 evaluation, added a complementary 2D visualization, and included DAAAM in the related work. (Secs. 2.2, 4.1, Appendix A.10, Fig. 11)

Key textual revisions are highlighted in blue in the updated manuscript, with the corresponding locations indicated above.

We appreciate the reviewers' time and consideration, and welcome any further questions or discussion.

AuthorsOpenReview ↗

Summary of Rebuttal and Final Remarks

Dear Reviewers, ACs, and SACs,

We sincerely thank all reviewers for their constructive feedback and thoughtful discussions. We have revised and clarified our manuscript through additional experiments and analyses. All revisions are highlighted in blue in the updated manuscript.

Rebuttal outcomes.

Our average reviewer score improved from 2.33 to 2.67 (from (2, 2, 3) to (3, 2, 3)).

  • Reviewer mjFh (2 → 3): Confirmed that our revisions substantially improved the paper and adequately addressed the main concerns.
  • Reviewer neBZ (3 → 3): Acknowledged that we had addressed the comments and remained positive about the paper.
  • Reviewer Q6LV (2 → 2): Acknowledged our responses and additional experiments, but maintained concerns about the general positioning of the method. We clarified that our claim is a construction-agnostic residency layer, rather than a performance-invariant plug-and-play method.

Our contributions.

For the Embodied AI and Robotics community, PORTER addresses a fundamental challenge in long-term embodied scene memory: how robots can preserve accumulated 3D scene knowledge over extended operation without keeping every full object payload in limited onboard memory. By decoupling persistence from residency and remaining agnostic to the underlying 3DSG builder and storage tier, PORTER provides an extensible foundation for scalable memory management across diverse embodied systems and deployment settings.

We sincerely appreciate the reviewers' time, expertise, and constructive engagement throughout the discussion process.

With sincere appreciation,

The Authors of Submission #23517

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.