- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This work addresses streaming memory management by proposing a memory eviction method based on future predictive utility and semantic stability. This paper also introduces a new benchmark for evaluating memory utility and predictive-state stability.
Critical strengths
- This work addresses an important problem: how to retain information that supports future prediction under a fixed memory budget. The problem is well motivated and practical.
- Evaluating the complete memory state after each candidate eviction is a promising approach to capturing the future utility of stored information, beyond relying solely on past observations or the properties of individual memory elements.
Critical weaknesses
- The proposed method relies on an offline utility bank constructed using future observations from the training distribution. During deployment, counterfactual utility is estimated by retrieving similar experiences from this bank. However, the relationship between retrieved experiences and future utility may not remain reliable under distribution shifts or in previously unseen domains. This assumption limits the practical applicability of the proposed method and requires further discussion or experimental validation.
- The improvements on Assembly101 are relatively small compared with the baselines. These results provide limited evidence of the effectiveness of the proposed method, particularly given its additional complexity and computational overhead. Stronger empirical evidence is needed to justify the proposed design.
- The methodology is difficult to follow because the motivation and role of each component are not clearly connected to the problem being addressed. The introduction of numerous technical details and new terms further obscures the overall design logic. The authors should explain the problem addressed by each component and establish a consistent line of reasoning that connects these components to the proposed method.
Questions for authors
Is it possible to evaluate the future utility of individual memory slots at test time, at least retrospectively using observed future data? Such an analysis could help demonstrate whether the proposed method preferentially retains high-utility memory elements and provide further insight into how the memory eviction mechanism works.
7 comments
Response to Reviewer VQop [2/3]
2. [W2] Improvement relative to complexity
We agree that the headline Assembly101 Action NLL reduction from 3.6315 to 3.6291 is modest given the additional storage and computation. We therefore tested whether SPACE's predictive gains persist on reserved recordings and additional datasets, whether simpler utility estimators can match those gains, and how much of the gain remains with a smaller Utility Bank.
a. Predictive gains on reserved recordings and additional datasets
On reserved Assembly101 recordings with the original thresholds and learned assets frozen, SPACE achieves a paired Action NLL gain of +0.001865 (participant-clustered 95% CI [0.000961, 0.002755]), with positive gains for 8 of 9 participants. SPACE also lowers mean Action NLL and yields a positive paired H64 gain over Reservoir on each additional dataset.
| Dataset | Mean Action NLL gain over Reservoir | Paired H64 gain (mean ± seed SD) |
|---|---|---|
| Breakfast | +0.0285 | +0.0328 ± 0.0091 |
| IKEA ASM | +0.0798 | +0.0877 ± 0.0051 |
| EPIC-KITCHENS-100 | +0.0230 | +0.0433 ± 0.0094 |
SPACE’s predictive gains extend beyond the original threshold-selection split and persist across three additional datasets. The full cross-dataset results are provided in Tables 8–10 of Appendix D in the revised PDF.
b. Comparison with simpler utility estimators
We trained Ridge (α=1), a 64/32-unit ReLU MLP, and histogram GBDT (80 rounds, up to 15 leaves) on the same training descriptors and counterfactual targets. With inherited guards, each estimator replaces the retrieved utility mean and retains bank-derived uncertainty, support, and the other gates. The bank-free variants use a fixed threshold on the predicted utility advantage. Both decision rules compute the candidate descriptors.
| Estimator and decision rule | Action NLL (mean ± seed SD) ↓ | NLL gain (×10⁻³) ↑ | H64 gain (×10⁻³) ↑ | Override rate (%) |
|---|---|---|---|---|
| Reservoir | 3.6423 ± 0.0228 | 0.0000 | 0.0000 | 0.000 |
| SPACE retrieval | 3.6405 ± 0.0211 | +1.7725 | +5.4061 | 1.022 |
| Ridge with inherited guards | 3.6428 ± 0.0234 | −0.5534 | −1.2835 | 0.747 |
| MLP with inherited guards | 3.6423 ± 0.0248 | +0.0101 | +0.4950 | 4.280 |
| GBDT with inherited guards | 3.6426 ± 0.0226 | −0.2836 | −0.2474 | 0.080 |
| Ridge with bank-free point decisions | 3.6658 ± 0.0236 | −23.4800 | −42.4820 | 8.203 |
| MLP with bank-free point decisions | 3.6533 ± 0.0214 | −10.9800 | −3.9740 | 31.590 |
| GBDT with bank-free point decisions | 3.6503 ± 0.0218 | −8.0570 | −13.6100 | 6.000 |
SPACE retrieval yields the largest mean Action NLL and H64 gains, while all three bank-free point variants increase Action NLL relative to Reservoir. The tested simpler estimators therefore fall short of reproducing SPACE’s predictive benefits, even when they retain the original guards.
c. Predictive gains and cost at smaller bank sizes
In a separate Assembly101 ablation, we formed nested bank subsets stratified by training participant and source policy (FIFO or Reservoir). We evaluated each bank size on the same 30 validation recordings from five participants with three matched policy seeds, keeping the predictor, thresholds, and slot count fixed. We report paired gains over Reservoir, tensor storage, and time per online policy update.
| Configuration | Bank tensors (MiB) | Action NLL ↓ | NLL gain (×10⁻³) ↑ | Participant 95% CI (×10⁻³) | H64 gain (×10⁻³) ↑ | Override rate (%) | Policy time (ms/update) |
|---|---|---|---|---|---|---|---|
| Reservoir | 0.0 | 3.6423 ± 0.0228 | 0.0000 ± 0.0000 | — | 0.0000 | 0.000 | 0.77 |
| SPACE, 12.5% bank | 92.5 | 3.6411 ± 0.0219 | +1.1732 ± 0.9945 | [0.6384, 1.7914] | +3.2648 | 0.630 | 7.87 |
| SPACE, 25% bank | 185.0 | 3.6404 ± 0.0227 | +1.9318 ± 0.4695 | [0.9171, 2.9450] | +4.2063 | 0.759 | 8.07 |
| SPACE, full bank | 740.0 | 3.6405 ± 0.0211 | +1.7725 ± 1.7283 | [0.0139, 3.4228] | +5.4061 | 1.022 | 9.66 |
The 25% bank cuts tensor storage by 75% (740 to 185 MiB) and policy time by 16.5% (9.66 to 8.07 ms per update) relative to the full bank. It retains 78% of the full bank's H64 gain and a positive Action NLL gain of +0.001932 over Reservoir.
A quarter-size bank preserves positive Action NLL and H64 gains while reducing storage and update time, providing a practical way to tune SPACE’s computation–storage trade-off.
Response to Reviewer VQop [3/3]
3. [W3] Clearer explanation in the revised PDF
Thank you for suggesting that we connect each component to the problem it solves. We have revised the Method section to clarify each component’s role in the decision to follow or override Reservoir.
- Evaluate candidate removals. The frozen predictor forecasts the futures supported by each resulting memory set.
- Screen for utility. The Utility Bank estimates which candidate changes are likely to improve prediction over the Reservoir action. The guards use retrieval support and uncertainty to screen these estimates.
- Stabilize predictions within a regime. The slow basis and basin prototype guide the choice among admissible changes toward a stable predictive state.
- Adapt at boundaries. When the current basin becomes unreachable, SPACE follows the Reservoir action and updates the basin prototype more quickly.
The revised presentation explains when SPACE follows Reservoir, what evidence warrants an override, and how the decision changes at a boundary.
4. [Q1] Retrospective utility of individual slots
Thank you for suggesting that we evaluate individual slots retrospectively with observed futures. We measure a slot’s predictive utility by evaluating how removing it affects future predictions given the rest of the memory.
At 960 sampled memory states from one policy seed, we enumerate all 17 removal actions at K=16, including discarding the incoming observation. We compare each candidate with the Reservoir action at the same state and score both using the same observed future. The cost equally weights training-normalized one-step and 32-step prediction terms, and the rollout branches share subsequent random choices. The override-level analysis uses the 209 executed SPACE overrides with complete oracle targets reported in Table 14 of the revised PDF. For each override, we compare the chosen action with the Reservoir action at the same memory state.
| Evaluation level | Diagnostic | SPACE | Reference or 95% CI |
|---|---|---|---|
| Candidate-level | Pairwise Sign Accuracy ↑ | 61.98% | 51.29% majority |
| Candidate-level | Hit@1 ↑ | 13.53% | 5.88% random |
| Candidate-level | Hit@3 ↑ | 28.29% | 17.65% random |
| Override-level | Beneficial Override Fraction ↑ | 58.85% | [55.06%, 62.99%] |
| Override-level | Actual Override Gain ↑ | +0.4166 | [0.1188, 0.9328] |
In the context of the remaining memory, SPACE identifies beneficial removals more accurately than the reference baselines, and its actual overrides improve retrospective predictive utility on average.
Response to Reviewer VQop [1/3]
We thank the reviewer for recognizing the importance of memory retention for future prediction and the promise of evaluating the complete memory state after each candidate eviction.
1. [W1] Reliability under distribution shifts
Thank you for raising the question of how the Utility Bank performs under distribution shift. We use future observations from training streams to construct utility targets. During evaluation, the controller queries the frozen bank using information available at each update.
a. Task and override outcomes under distribution shift
We kept the bank, predictor, and thresholds fixed across all four settings. An assembly configuration refers to the toy being assembled, identified by its Assembly101 toy ID. A configuration is seen if the Utility Bank contains training examples with that toy ID and unseen if the ID is absent from the bank. The observation stride determines how often the controller receives an observation. At native stride, the controller receives every observation in the standard input stream. At doubled stride, it receives every other observation from that stream, halving the input rate. We count prediction horizons in observations received by the controller.
Action NLL gain measures the paired reduction in loss over the full stream. Realized override utility measures the hindsight gain of the action SPACE actually chose over the Reservoir action at the same memory state, using equal weights on normalized one-step and 32-step prediction costs. With the bank, predictor, and thresholds frozen, SPACE retains positive Action NLL gains on unseen configurations at both strides, supporting generalization to the tested shifts.
b. Utility ranking and gate performance under the same shifts
To test whether the bank continues to rank useful actions under shifts, we sampled up to eight Reservoir decision states per recording and seed and scored all 17 actions using observed futures. Sign accuracy checks the direction of gain over Reservoir. Spearman measures agreement with the hindsight ranking, and Hit@k checks whether the hindsight-best action appears among the top k predictions. Gate precision is the fraction of admitted alternatives with positive hindsight gain.
Mean utility-ranking correlations remain around 0.315–0.320 across all four conditions. On unseen configurations, doubling the stride raises sign accuracy from 60.86% to 65.41%, while gate precision stays near 55.5% and Hit@1 changes from 15.18% to 13.74%. Higher sign accuracy can result from correctly identifying more harmful candidates, while Hit@1 depends on ranking the best removal first. These diagnostics capture different aspects of utility estimation, alongside a positive stream-level predictive gain under the combined shift.