acceptodds
Under review as a conference paper at ICLR 2027

Persistent-State Activation Beyond Utility: Scale Representation and Evidence for Non-Compensatory Control

Abstract

Persistent memory creates a control problem that relevance alone does not solve: evidence can remain semantically useful after an update makes it inadmissible for the current answer. We separate utility from authority. A fixed finite penalty on an unnormalized utility scale cannot guarantee exclusion under arbitrary positive-affine reparameterization, while rank-normalized finite compensation is scale-invariant and is therefore the appropriate empirical alternative. Our originally frozen TIDE/LongMemEval primary was quarantined before HELDOUT inference because its operationalizations failed construct/split validity; we do not report that matrix as confirmation. We instead report a post-primary replacement diagnostic on Memora Remembering (Uddin et al., 2026), where deletion/update state supplies item-level forgetting evidence. On Qwen3-8B (Yang et al., 2025), DEV selected λ = 0.6 from a ten-point SOFT family. Across 185 HELDOUT units in 28 dependency clusters, HARD has 4.04% balanced construct error versus 19.66% for locked SOFT; the paired difference is 15.61 points with a 20,000-replicate cluster-bootstrap 95% interval of [13.07, 18.76]. This is forgetting-sensitive answer/admission evidence, not an independent system-harm endpoint or completion of the original six-cell macro. The replacement scorer and utility proxy were finalized during replacement development, so the cell is operator-comparison evidence rather than prospective causal confirmation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
persistent memory, admissibility control, memory governance, stale evidence, memory updates, hard gating, soft penalties, forgetting, long-context reasoning, neurosymbolic AI, formal reasoning, Qwen3-8B, Memora Remembering

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 4 reviews, above 0% of the others.
Reviewer_Qe6cOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper distinguishes how useful a memory is from whether it is currently permitted for use. HARD removes memories labeled inadmissible before ranking the remaining candidates. SOFT keeps the candidates and subtracts a penalty from the scores of inadmissible memories. The theoretical analysis concerns fixed penalties on unnormalized utility scores under rescaling; the experiments use a normalized recency score. Both controllers receive the labels identifying which memories should no longer be used.

After excluding planned TIDE/LongMemEval evaluations that failed its protocol checks, the paper reports a replacement comparison using Qwen3-8B on Memora Remembering. The authors select a SOFT penalty from ten values using 15 development units, then evaluate on 185 held-out units. HARD achieves 4.04% balanced construct error, compared with 19.66% for SOFT. This error averages the missed-current-value rate and the forgotten-value leakage rate, based on whether the relevant value strings appear in the answer. The authors limit the result to the tested controllers, supplied labels, model, benchmark, and scoring procedure.

Critical strengths

The paper clearly separates a memory's usefulness from its current admissibility. This distinction provides a concrete question for evaluating how persistent memory systems handle updates, deletions, and revocation: whether information that should no longer be used can still influence an answer because it receives a high retrieval score.

Critical weaknesses
  1. Insufficient theoretical and algorithmic novelty. Proposition 1 shows that a fixed additive penalty can be overcome by a sufficiently large positive rescaling of utility scores. Its proof reduces to choosing a scale for which a(Uj−Uk)>λa(U_j-U_k)>\lambda. This elementary scale-dependence observation is not developed into a substantial theoretical advance, and it does not establish a performance advantage over the normalized SOFT controller used in the experiment, which already removes that particular scaling defect.

    With inadmissibility labels supplied, HARD filters candidates and then ranks the remainder. A scalar rule with an appropriate admission threshold can reproduce its eligible set and ranking. The manuscript does not establish sufficient new algorithmic capability or theoretical insight beyond applying established hard-constraint and penalty-based control ideas to persistent memory. The distinction between usefulness and permission is relevant, but the formulation presented here does not provide sufficient novelty to support acceptance.

  2. The empirical study does not establish a sufficiently substantive new finding. The completed performance evidence consists of one Qwen3-8B comparison on Memora Remembering, using 15 development units to select among ten penalties and 185 held-out units for evaluation. The balanced construct errors of 4.04% for HARD and 19.66% for SOFT require examination of their components: current-value recall differs by only 2.40 percentage points (91.92% versus 89.52%), whereas forgotten-value absence differs by 28.83 points (100.00% versus 71.17%). Most of the aggregate improvement therefore comes from avoiding specified forgotten values after the corresponding memories are removed. This score is directly coupled to the filtering operation and provides limited independent evidence of an improvement in answer quality.The comparison also lacks the controls needed to explain the observed gap. The provided PDF does not specify whether SOFT fills the available Top-K positions or rejects candidates below a threshold. If it fills them when permitted candidates are insufficient, forbidden memories can enter even when ranked below every permitted memory. The experiment does not separate this filling behavior from effects of prompt length, memory count, position, and distractor composition. The study also provides no completed cross-model or cross-benchmark validation and does not quantify uncertainty from development-set selection. The reported result supports a narrow comparison of the tested implementations, but does not supply a compelling empirical contribution that compensates for the limited theoretical and algorithmic novelty. I therefore recommend clear rejection; addressing these concerns requires substantial additional research and validation.

Questions for authors
  1. Which memories enter the prompt? Does SOFT continue filling the available Top-K positions when fewer than K permitted candidates exist? Does it use a score threshold or another rejection rule? Please give the exact admission procedure and report, for each unit, the numbers of permitted and forbidden candidates, the numbers of each actually injected, and the resulting prompt length. If these details are in supplementary artifacts, please identify the exact files.

  2. Comparison with a thresholded scalar rule. With R∈[0,1]R\in[0,1], q∈{0,1}q\in\{0,1\}, and λ>1\lambda>1, the condition R−λq≥0R-\lambda q\geq0 admits exactly the q=0q=0 candidates and preserves their ranking by RR. This reproduces HARD's eligible set and ranking. The manuscript acknowledges that scalar thresholds can implement hard feasibility. Please clarify what the tested SOFT admission rule adds to this comparison, and distinguish increasing the penalty while still filling Top-K from using a threshold that rejects forbidden candidates.

  3. Effect of context length and composition. HARD can supply fewer memories and a shorter prompt than SOFT. Can controlled comparisons quantify how much of the performance difference depends on the number, position, and length of the supplied memories or on interference from distractors? Please report these context differences alongside the results. This would help assess the explanation of the gap beyond the overall effect of the two tested controllers.

  4. Stability of penalty selection. The complete DEV curve and the deterministic rule selecting λ=0.6\lambda=0.6 are already reported. How stable are the selected penalty and the HARD–SOFT difference when the small development sample changes? Please assess selection uncertainty using a procedure that keeps related units grouped, such as alternative development splits or nested selection. The held-out confidence interval conditions on the selected penalty; it does not quantify uncertainty from choosing among ten values using 15 DEV units.

  5. Practical meaning of the reported score. The paper explicitly limits its endpoint to current-value presence and forgotten-value absence. Can the authors provide a representative analysis of model outputs showing how changes in this score correspond to correct use of current information and avoidance of forgotten information? This would help assess the meaning of the improvement within the benchmark, beyond the direct benefit of removing memories labeled inadmissible from the input.

  6. Scoring units without forgotten-value targets. How are units without materialized forgotten-value strings scored in the main 185-unit analysis? Please specify the empty-target convention and how current recall, forgetting absence, and balanced construct error are aggregated across units. The reported 167-unit sensitivity analysis, with HARD at 1.78% and SOFT at 19.08% error, is useful; this question concerns the definition of the main-cohort score.

Reviewer_eHiJOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper distinguishes two aspects of an item stored in an LLM’s memory system: authority (whether it is still admissible after an update or deletion) and utility (how useful it is for answering). It compares a hard controller, which drops inadmissible items before ranking, with a soft controller, which keeps them but subtracts a penalty λ from a rank-normalized recency score. Its one theoretical result is that a fixed penalty on raw scores can be overridden by rescaling those scores, which motivates the normalized soft baseline. The preregistered experiment (TIDE and LongMemEval across three models) was abandoned after audits found a mislabeled adapter and a broken dev split. In its place, the paper reports a replacement experiment using Qwen3-8B on Memora’s Remembering task. Hard exclusion reduces balanced error from 19.66% to 4.04%, primarily by reducing forgotten-value leakage while maintaining current-value recall. Both controllers receive correct admissibility labels. The authors acknowledge that the utility proxy was developed during replacement execution and the scorer finalized after dev outputs existed, that hard exclusion’s perfect forgetting score is partly expected because forgotten values are removed from the prompt, and that they cannot separate gating effects from changes in context composition.

Critical strengths

None

Critical weaknesses

The contribution appears limited. The theoretical result follows immediately from rescaling a score while holding its penalty fixed. The experiment supplies correct inadmissibility labels and shows that removing forbidden information reduces its appearance in answers compared with merely lowering its retrieval rank—an unsurprising result whose architectural significance is unclear. The comparison also confounds exclusion with changes in context composition. With an average of 12.37 candidates per unit and a 12-item context budget, it is unclear how often the soft penalty actually changes which items enter the prompt rather than merely reordering them. Empirical support is restricted to one model and benchmark, with only 15 development examples for penalty selection and a scorer finalized after development outputs existed. These limitations leave little evidence of a substantive advance beyond straightforward filtering.

Questions for authors

How often does SOFT actually exclude inadmissible items? If rarely, the comparison is just deleted items in the prompt vs. not.

Does the model see the label? Are inadmissible items marked as such under SOFT? If not, why isn’t “include everything but annotate deleted items” the baseline?

Reviewer_d6F4OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies memory in long-running agents, focusing on memory updates and authorization.

Critical strengths

None

Critical weaknesses

The paper does not appear to make any important contribution to science.

Questions for authors

None

Reviewer_CAxEOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether persistent-memory control should separate memory authority from retrieval utility. It compares hard exclusion with a rank-normalized finite-penalty soft controller and presents a rescaling argument against fixed penalties on unnormalized utility scores.

After the original TIDE/LongMemEval evaluation was abandoned due to protocol issues, the authors conducted a replacement experiment on Memora Remembering with Qwen3-8B. hard achieves substantially lower forgetting-sensitive error than soft, although the authors limit this finding to the tested setting.

Critical strengths
  1. The paper clearly separates memory authority from retrieval utility without overstating the limitations of scalar controllers. This makes the scope of its theoretical claim relatively precise.
  2. The experimental reporting is careful. The authors explain why the original evaluation was abandoned, preserve a heldout split for the replacement experiment, and report cluster-level uncertainty estimates.
Critical weaknesses
  1. The main comparison is heavily confounded by the experimental design. Hard removes inadmissible memories, while soft only penalizes their scores without an exclusion threshold. With a 12-candidate budget and roughly five current candidates per question, soft may still admit inadmissible memories regardless of the penalty. The forgetting metric directly favors hard in this setting. Without a stronger baseline, the reported performance gap provides limited evidence of a meaningful architectural contribution.
  2. The novelty and empirical contribution are insufficient. Proposition 1 is a straightforward rescaling argument, and separating authority from utility is already well established. After the original evaluation was abandoned, the evidence rests on a single post-primary Qwen3-8B experiment with known authority labels. I do not see a sufficiently novel theoretical result or a compelling empirical finding to justify acceptance.
Questions for authors
  1. How many heldout units have fewer than 12 admissible candidates, and how much of soft's forgetting error occurs in these cases?
  2. Would the main result remain meaningful against a normalized controller that can leave prompt slots empty or apply an explicit admission threshold?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.