- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
MemSAGE addresses contextual-integrity violations in persistent-memory LLMs by learning context-conditioned permission and demand subspaces on a Stiefel manifold. It derives asymmetric geometric scores for each retained memory, then modifies attention to suppress norm-violating memories and amplify task-obligatory ones without updating the backbone. Experiments on CIMemories and CI-RL with four LLMs report lower disclosure violations, improved completeness.
Critical strengths
- The paper introduces a technically distinctive formulation: nested permission-demand Stiefel geometry is converted into asymmetric, memory-specific attention control, providing a more granular alternative to prompt constraints or hard retrieval filtering.
- The three-stage design is coherent and modular: contextual framing, spectral beamforming, and token-level steering connect an explicit contextual-integrity model to a frozen-backbone inference intervention.
Critical weaknesses
- The rationale for using the Stiefel manifold is insufficiently developed. The paper explains the benefits of orthonormal frames, but does not clarify why the present problem specifically requires a Stiefel-manifold formulation. It should discuss plausible alternatives and explain why Stiefel geometry is preferable to other orthogonalization or subspace-learning approaches.
- The performance of the permission and demand classifiers is not reported. Equation 6 directly constructs supervision from and , yet the paper provides no classification accuracy, recall, or confusion matrices for these two prediction tasks.
- Table 5 does not establish the necessity of the Stiefel geometry. It only replaces the spectral gate with MLP and bilinear readouts, while retaining the same Stiefel-frame construction, training objectives, and attention-steering pipeline. Thus, it shows only that the spectral readout outperforms two simpler scoring heads, not that orthogonality, principal-angle information, or the Stiefel manifold is necessary.
- The construction of the training trajectories used in Equation 7 is unclear. Computing appears to require query representations from the complete generation trajectory. The paper does not specify which model and steering configuration generate these trajectories, or whether reference responses or future generation states are used during training.
Questions for authors
See weakness
6 comments
Response to Reviewer vLu2 (2/2)
2. Regarding the performance of the permission and demand classifiers (W2):
We appreciate your feedback on this aspect. We evaluate the permission and demand predictions against the ground-truth contextual labels using AUROC, accuracy, recall, and precision, with results reported in Table R2. For recall and precision, the positive class is norm-violating memories for permission prediction and task-obligatory memories for demand prediction.
Table R2. Standalone performance of the permission and demand classifiers.
| Classifier | AUROC | Accuracy (%) | Recall (%) | Precision (%) |
|---|---|---|---|---|
| Permission | 0.811 | 75.5 | 81.7 | 80.9 |
| Demand | 0.880 | 80.6 | 76.3 | 83.5 |
We highlight three observations:
- Permission scores effectively identify norm-violating memories. The permission classifier achieves a recall of 81.7%, indicating the ability of the learned permission scores to identify memories that warrant suppression. This is critical for contextual integrity since missed norm-violating memories may lead to inappropriate disclosure during generation.
- Demand scores reliably identify task-obligatory memories. The demand classifier achieves an accuracy of 80.6% and a precision of 83.5%, supporting selective recognition of task requirements. This is important for demand-aware steering, as mistaking unnecessary information for task-obligatory content can divert emphasis from information genuinely required for task completion.
- Permission and demand scores provide graded discrimination between memory categories. Their AUROC values of 0.811 and 0.880 indicate that both scores meaningfully rank memories according to their contextual labels. This extends their utility beyond binary decisions at a particular threshold, supporting the permission–demand geometry underlying MemSAGE’s memory steering.
We believe these results provide a more comprehensive understanding of the effectiveness of MemSAGE’s permission and demand predictions. We will incorporate these results and the corresponding analysis into the revised manuscript.
3. Regarding the construction of training trajectories (W4):
Thank you for your feedback. We provide the clarifications as follows:
(1) Trajectory generation: We generate the reference responses offline using the same frozen backbone, without applying MemSAGE steering. For each training context, we remove norm-violating memories from the generation prompt and explicitly instruct the model to incorporate task-obligatory memories based on the ground-truth labels. These responses are generated once using only the training profiles and remain fixed throughout training.
(2) Query extraction and alignment supervision: We append each reference response to the original complete prompt containing all memory entries and perform a teacher-forced forward pass through the frozen backbone. We extract queries from the response positions and keys from the memory spans to compute the alignment objective in Equation 7. The standard causal mask prevents each position from attending to subsequent response tokens. These response queries serve only as training supervision for the demand-conditioned projection. During inference, the permission scores and projections are computed from the context and memory representations without access to reference responses or future generation states.
We will clarify the reference-response construction and alignment supervision in the revised manuscript.
Dear auhtor,
Thank you for the response. However, my main concerns are not addressed:
- W1: The rebuttal only restates generic benefits of orthonormal frames, which I had already acknowledged. It still does not justify why the Stiefel manifold is required over standard alternatives such as Grassmann parameterization, QR/Cayley orthogonalization, or soft orthogonality constraints. Neither the paper nor the rebuttal compares against any of them. The core design choice remains unjustified in both sufficiency and necessity.
- W3: Table R1 does not support the authors’ claim. The variants are clearly degenerate: the unnormalized one reaches only 23.45% Completeness, worse than Vanilla (37.65%, Qwen3-8B in Table 1). At most, the results show that the pipeline was built around orthonormality. They do not show that Stiefel geometry is necessary.
- Revision: The authors promise revisions but have not uploaded a revised PDF, so none of the claimed changes can be verified.
Given these unresolved issues and the unrevised manuscript, I retain my original score of 2.
Response to Reviewer vLu2 (1/2)
Thank you for your careful assessment and constructive suggestions. We provide the following clarifications to address your concerns:
1. Regarding the rationale and empirical justification for Stiefel geometry (W1, W3):
We appreciate your insightful feedback. In response, we provide the following clarifications and supporting evidence:
(1) Design rationale for Stiefel geometry: We would like to clarify that the Stiefel formulation is essential to the spectral alignment, nested geometry, and memory steering of MemSAGE. Specifically:
(2) Controlled evaluation of the Stiefel geometry: To directly examine the necessity and effectiveness of the Stiefel formulation, we conduct complementary experiments under the following setup:
Setup: We compare MemSAGE with two variants: unnormalized orthogonalization, which retains mutually orthogonal columns while allowing their norms to vary, and unconstrained subspace construction, which directly uses the hypernetwork outputs without orthogonalization or normalization. Each variant applies the same modification to the context and memory frames and the factor constructing the nested demand frame. All other training and steering settings remain unchanged.
We highlight two observations:
We believe these clarifications and experimental results provide a more comprehensive understanding of the rationale for and effectiveness of the Stiefel formulation in MemSAGE. We will incorporate these discussions and additional results into the revised manuscript.