acceptodds
Under review as a conference paper at ICLR 2027

PolicyMem: Geometric Policy Memory for LLM Governance

Abstract

Effective LLM governance extends beyond detecting unsafe responses to guiding their correction through closed-loop assessment, revision, and verification. Learning-based guards provide strong semantic discrimination but often leave policy knowledge implicit in trained parameters, limiting its explicit reuse across stages. Programmable guardrails make policies explicit through prompts and workflows, but rely heavily on manual engineering to operationalize them. This motivates explicit policy representations that are both **learnable and reusable** across governance operations. To this end, we introduce **PolicyMem**, a geometric policy memory that compiles natural-language policies into learnable low-rank subspaces for reuse across governance stages. A policy assessor projects query-response representations onto these subspaces, producing evidence that directly mediates safety judgments and policy attribution. A response rewriter uses the implicated policies to revise unsafe responses, with the same assessor re-evaluating the revised responses. Beyond inference, the same memory constructs training data from model-generated candidates to supervise the rewriter without external teacher calls. Across five widely used benchmarks, PolicyMem achieves the best average unsafe-behavior detection performance among representative baselines while supporting effective policy attribution, rewriting, and post-intervention assessment through the shared policy memory.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Safety, LLM Governance

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qJLXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PolicyMem represents each safety policy as a low-rank subspace in a shared geometric space. Queries and responses are scored by their projection energy onto these subspaces, which drives the safety verdict and policy attribution. Flagged responses are rewritten using the implicated policies and rechecked against the same memory. On five benchmarks, the 3B variant reaches the highest average Unsafe-F1 among the baselines.

Critical strengths
  • The question is well motivated: expressing gradient updates in a vocabulary people can inspect, and using that vocabulary for control, is a worthwhile direction.

  • The method is carefully built. Fixing the LoRA A factor to make per-example signatures comparable is well justified.

Critical weaknesses
  • Detection gains are not explained by the method. The method does not show why it should intercept more. The baselines are off-the-shelf, while PolicyMem is trained per benchmark, so the comparison may reflect training data rather than the memory design.

  • Rewriting lacks a fair LLM baseline. Qwen3-4B and Qwen3-8B are prompted without policy feedback, but there is no same-backbone comparison with and without the policy descriptions, and no baseline that places the policy text directly in the prompt.

  • Rewrite utility is not measured. "Safe helpfulness" judges only the legitimate part of a request. Task completion and over-refusal rate are not reported, so a safe but unhelpful rewrite can score well.

  • Rewriter training data quality is unknown, and baselines may be weak.

  • Expansion without training is not demonstrated. The K=0 result (92% detection) is not zero-shot in the strict sense, since the encoder and verdict module were trained on the original five benchmarks.

Questions for authors
  1. see weakness parts
  2. I am wondering, if everything is unchanged except the policy memory bank/subspace, how would a LLM-based policy processer can perform in the system? What is the advantage of the proposed policy memory v.s. LLM-based policy analyzer? Please demonstrate this from methodology perspective and experiment performance perspective.
6 comments
AuthorsOpenReview ↗

Response to Reviewer qJLX (Part-1)

We thank the reviewer for recognizing the importance of making learned safety behavior interpretable and actionable. We address your questions and concerns point by point below.

W1: Explaining detection gains beyond the use of benchmark-specific training

Thank you for raising the question of how to distinguish the benefits of PolicyMem's representation from those of task-specific supervision. Our original evaluation follows the released-checkpoint, cross-benchmark protocol used in safeguard research [1,2,3]. These checkpoints are safety-specialized models trained on substantial moderation corpora, making them meaningful, ready-to-use baselines rather than untuned general-purpose LLMs. To strengthen this comparison, we additionally adapt three established guards to our training source, highlight the single co-trained PolicyMem governor evaluated across all five taxonomies, and directly examine what the governance-specific modules learn with backbone adaptation frozen.

Task-adapted guard baselines. Granite Guardian, WildGuard, and MD-Judge are each additionally fine-tuned on the same adaptation sample with a common LoRA configuration. All are evaluated on the same five test splits, using their official prompt formats and output parsers under a common per-case input budget.

Table 1R: Released and task-adapted guard comparisons (macro Unsafe-F1).

ModelParametersReleased checkpointTask-adapted/PolicyMem
granite-guardian-3.1-8b8B.8356.8741
wildguard-7b7B.8633.8684
MD-Judge-v0.2-7B7B.8576.8411
PolicyMem0.5B-.8727
PolicyMem3B-.8874

Three findings are notable. First, the 0.5B PolicyMem exceeds all three released guards with roughly 14 times fewer parameters than a 7B guard. Second, the 3B PolicyMem reaches .8874, exceeding the strongest additionally adapted guard (.8741) while using 2.3-2.7 times fewer backbone parameters. Third, a single 3B co-trained governor reaches .8799 across all five benchmarks, without selecting a taxonomy-specific model at inference time. Thus, strong performance is not confined to separate per-benchmark specialists: a shared geometric policy memory remains effective across heterogeneous taxonomies.

We also note that released safety checkpoints are not necessarily unexposed to related benchmark distributions: MD-Judge's model card lists BeaverTails among its training data, and WildGuard is trained using WildGuardTrain, which is related to the WildGuardMix evaluation. The revised manuscript distinguishes released from task-adapted guard baselines in Section 4.1 and Table 1.

Representation beyond backbone adaptation. We further freeze the backbone LoRA adapter and train only the governance-specific modules, including the case projection, subspace compiler, and soft-token projection. The adapter still participates in the forward pass but receives no updates.

Table 2R: Learning policy evidence with the backbone adapter frozen. Hit@1 is measured against the 77-policy union memory of this control's own reference run.

MetricPooled-5 referencePooled-5, adapter frozenPooled-4 referencePooled-4, adapter frozen
Trainable parameters12.74M3.94M12.74M3.94M
Policy attribution Hit@1.6959.6038.7751.6835
Mean off-diagonal slot overlap.1879.1984.1952.1995

With only 31% of the reference trainable parameters, the governance modules retain 86.8% and 88.2% of the corresponding attribution Hit@1, while maintaining similar subspace separation. This demonstrates that the explicit, addressable policy-evidence structure is learned by the governance modules even without further adapting the backbone. Together, the task-adapted guard comparison and frozen-adapter control provide complementary evidence for PolicyMem's strength as both a detector and an operational policy representation.

AuthorsOpenReview ↗

Response to Reviewer qJLX (Part-2)

W2: Fair rewriting comparisons with and without policy feedback

Thank you for requesting a same-backbone comparison that directly tests the role of policy guidance. We evaluate all feedback variants on the same fixed case rewriting cohort and independent judges (o3 and GPT-5.2). Only the policy feedback supplied to the rewriter changes. This isolates the practical value of targeted attribution from differences in model capacity, training, or evaluation cases.

Table 3R: Policy-feedback ablation with a fixed rewriter (Eq. 23 joint good-rewrite rate).

Policy feedback supplied to the same rewriterJoint good-rewrite ratePrompt tokens
Generic safety instruction.7395361 (0.88x)
Full 77-policy descriptions.73451,906 (4.67x)
All policies in the case's taxonomy.7568653 (1.60x)
PolicyMem Top-3 attribution.7677408 (1.00x)

Focused PolicyMem feedback achieves the best rewriting outcome with a compact 408-token prompt. Compared with providing all policies, it raises joint good-rewrite success by 3.32 percentage points while using 4.67 times fewer prompt tokens. It also improves over the shorter generic instruction by 2.82 points. The advantage therefore is not explained simply by adding more policy text or by shortening the prompt. It comes from selecting case-relevant policy evidence, the very capability supplied by an individually addressable memory.

W3: Measuring task completion and over-refusal

We agree that governance should be assessed not only by safety but also by its effect on legitimate user requests. We therefore evaluate the deployed governor end-to-end on XSTest (450 prompts) [4] with independent judges. The 250 adversarially benign prompts, deliberately phrased to resemble unsafe requests, directly probe over-refusal and task-completion failures.

Table 4R: End-to-end utility on 250 adversarially benign XSTest prompts.

MetricQwen2.5-3BWith PolicyMem governorChange
Over-refusal, full or partial.1280.1280.0000
Task completion.8200.8200.0000

The governor flags only one response among the 450 prompts, and that flag falls in the benign subset: a 0.4% false-flag rate over its 250 prompts. On these challenging benign requests, PolicyMem introduces no measured increase in over-refusal and no measured decrease in task completion. On the separate 200 unsafe prompts, the evaluated delivery pipeline yields no observed harmful deliveries. These results complement the paper's safe-helpfulness metric with deployment evidence: PolicyMem preserves benign-task utility while maintaining safe delivered outputs on this held-out evaluation.

W4: Rewriter training-data quality and baseline strength

Thank you for asking for greater transparency about the rewriter's supervision and for strong teacher-based comparisons. PolicyMem constructs supervision without externally generated teacher targets. Its frozen assessor evaluates self-generated candidate revisions against the configured policy memory, selecting SFT targets and preference pairs using the policy-grounded safety-confidence criteria in Section 3.6.

Table 5R: Construction of PolicyMem's rewriter supervision.

StageNumber
Unsafe target responses with PolicyMem Top-3 feedback10,000
First-round candidate rewrites (4 per target)40,000
Second-round candidate rewrites40,000
Candidate rewrites selected for SFT3,558
Chosen/rejected preference pairs used for DPO912

This procedure is policy-grounded and auditable: each selected revision is linked to its originating case, attributed policy feedback, and assessor-based acceptance criterion. After assessor-filtered SFT, memory-guided DPO further favors revisions that better satisfy the configured policies, without external teacher calls for training-target generation. For a strong comparison, we evaluate a GPT-4o-teacher-supervised rewriter using the same target contexts, with outputs scored by the independent judges.

Table 6R: Independently judged rewriting quality.

SupervisionSafetyInformationHelpfulnessEq. 23 joint
GPT-4o teacher3.803.482.83.6994
PolicyMem supervision3.983.502.84.7677

PolicyMem improves the joint good-rewrite rate by 6.83 percentage points over the GPT-4o-teacher baseline, with stronger judged safety and comparable, slightly higher information preservation and helpfulness. This indicates more policy-compliant corrections without sacrificing legitimate content. The comparison also involves approximately $260 in external teacher API calls for the GPT-4o supervision arm versus $0 in external teacher calls for PolicyMem's self-generated supervision; this comparison concerns teacher API expenditure, not total training compute.

AuthorsOpenReview ↗

Response to Reviewer qJLX (Part-3)

W5 & Q: New-policy generalization and comparison with LLM-based policy analyzers (Part-1)

Thank you for connecting the new-policy question to the role of an LLM-based policy processor inside the complete governance system. PolicyMem's central design is to externalize natural-language policies as learnable, individually addressable geometric memory slots. Once compiled, these slots provide the same policy-level evidence for detection, attribution, targeted rewriting, and post-rewrite reassessment. We test this operational advantage through description-only onboarding, controlled initialization, trained text-reader comparisons, scalability measurements, and a direct in-system replacement.

Description-only onboarding of unseen policies. We use zero-shot onboarding to mean inserting a new policy into an already trained governor without target-policy labels or gradient updates, not training the governor itself without safety data. In the held-out Aegis 2.0 experiment, the target taxonomy and its labeled cases are excluded from training. Compiling its policy descriptions alone yields .8048 Unsafe-F1 on cases with zero target-policy labels.

We independently repeat the insertion protocol with 13 held-out WildGuardMix policy descriptions. Without any new gradient updates, PolicyMem achieves .7153 Unsafe-F1 and .9254 max-energy AUROC. The same trained memory writer can therefore make previously unseen policy descriptions operational at insertion time, before policy-specific fine-tuning.

What text-grounded compilation contributes. To isolate initialization from subsequent adaptation, we keep the geometry, subspace rank, replay/alignment procedure, policies, and evaluation cases fixed, changing only whether new slots are compiled from their natural-language descriptions or initialized randomly.

Table 7R: Text-grounded versus random policy-slot initialization.

InitializationText-compiled anchorRandom anchor
New-taxonomy Unsafe-F1.8701.2361
New-policy attribution Top-1 / mAP.7208 / .7730.1193 / .2660
Incumbent Unsafe-F1.8711.8696

With the same subsequent alignment, text-grounded initialization improves new-taxonomy Unsafe-F1 by 63.4 percentage points and new-policy attribution mAP from .2660 to .7730, while preserving incumbent detection performance. Combined with the label-free insertion results, this isolates a key advantage of the memory-write operation: policy descriptions provide useful initial representations that can be further refined with limited supervision.

Comparison with trained LLM policy analyzers. We implement two policy-text readers with the same backbone family, LoRA setup, pooled policy training collection, labels, and training-step budget: a list reader processing policy texts together and a block-isolated reader scoring policies individually. Both can accommodate new descriptions, but they must re-encode policy text for each case. PolicyMem instead compiles each description into a cached 256 x 8 subspace and reuses inexpensive numerical reads.

Table 8R: Geometric policy memory versus trained policy-text readers.

PropertyList readerBlock-isolated readerPolicyMem
Top-1 changes after policy-list permutation28.3%5.5%0.0%
Top-1 changes after mid-list policy insertion38.3%5.1%0.22%
New policy description at inferenceRe-encode per caseRe-encode per caseCompile once, then cache
Input tokens per case, 77 / 1,077 policies1,939 / 25,0941,856 / 24,011208 / 208
Latency at 77 policies (H100, batch 8)67.2 ms69.8 ms24.9 ms

PolicyMem requires 24.9 ms per case, compared with 67.2 ms and 69.8 ms for the trained readers. Scaling from 77 to 1,077 policies increases PolicyMem latency only from 24.9 to 28.5 ms (+14.5%), while its per-case text input stays at 208 tokens. The readers, by contrast, process 25,094 and 24,011 policy-text tokens per case at 1,077 policies. PolicyMem thus avoids repeated full-corpus text encoding, while its cached per-policy reads are independent of policy-list order. Under the measured insertion experiment, only 0.22% of top-1 attributions change, even as new slots become eligible for the top rank.

Reviewer_TqEbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PolicyMem addresses how to represent safety policies explicitly so that the same representation can support response assessment, policy attribution, corrective rewriting, and subsequent reassessment. Each natural-language policy is encoded into an anchor and compiled into a low-rank subspace with an orthonormal basis. A query-response pair is encoded into a normalized vector, and its squared projection onto each policy subspace produces a policy-specific evidence score.

The binary verdict module receives only a seven-dimensional, permutation-invariant summary of these evidence scores. The indexed scores also rank implicated policies, whose natural-language descriptions condition a separate response rewriter. Rewritten responses are assessed by the same assessor, with bounded retries and a refusal fallback. The assessor is trained using verdict supervision, policy alignment, policy-level supervision, and subspace-overlap regularization. Its judgments subsequently select self-generated rewrites for supervised fine-tuning and preference optimization, without external teacher-generated training targets (Section 3.6). New policies are introduced through description-based slot insertion followed by alignment with labeled examples and replay (Section 3.7).

Experiments use five safety benchmarks to evaluate detection, attribution, rewriting, sequential policy onboarding, and held-out-taxonomy coverage. Additional analyses examine efficiency, memory organization, hyperparameter sensitivity, and conditional geometric properties.

Critical strengths
  1. The verdict receives case-dependent information only through policy energies, while the indexed energies select policy descriptions for rewriting (Sections 3.3-3.5, Eqs. 9-12). This creates a concrete connection between assessment and intervention that is valuable for systems requiring addressable policy feedback. The separation between fixed-slot edits and learning updates also makes the limits of modularity explicit (Appendix A.4).

  2. The 3B specialist achieves average Safe-F1/Unsafe-F1 of 0.9238/0.8874, and the co-trained model achieves 0.9159/0.8799 across five benchmarks (Table 1). Independent judge evaluation reports a joint good-rewrite rate of 0.7677, compared with 0.6994 for the GPT-4o-teacher baseline (Table 4). These results support the practical promise of the implemented system, even though the source of its advantage and full-loop reliability require stronger validation.

Critical weaknesses
  1. The principal comparisons mix differences in representation, supervision, adaptation, and training procedures, while the available ablations do not fully control these factors (Section 3.6; Appendices B.2-B.3 and C.6). Resolving this requires substantive matched training and intervention experiments. It cannot be repaired by a clearer description alone, because the results could change how much of the contribution is attributable to geometric policy memory.

  2. TExternal judges evaluate single-pass rewrites, whereas the full retry loop is evaluated through the assessor that also supplies training supervision (Appendices B.3.3 and C.4). Independent evaluation of final delivery safety and legitimate usefulness, including missed detections, is needed to establish the proposed governor's end-to-end behavior. This requires additional evaluation and could materially change the apparent reliability of the system.

Questions for authors

The contribution of geometric memory is not sufficiently isolated. PolicyMem receives benchmark-specific training and policy-level supervision (Section 3.6; Appendix B.3.2), whereas the general-purpose Qwen baselines are prompted without task-specific fine-tuning, and the guard baselines use released checkpoints (Appendices B.2-B.3.1). These comparisons establish performance against the evaluated systems, but do not isolate the benefit of the proposed representation. Appendix C.6 provides relevant controls, including an anchor-based MLP result of 0.7765 versus 0.8799 Unsafe-F1 and trained text readers in Figure 11. Nevertheless, it does not specify enough about the MLP's case-policy interaction, supervision, capacity, or tuning to establish a matched comparison, and Figure 11 addresses attribution and permutation sensitivity rather than the complete detection-rewriting pipeline. A comparably trained binary/multilabel predictor and matched policy-conditioned rewriting controls are needed to substantiate the central attribution of gains to geometric memory.

4 comments
AuthorsOpenReview ↗

Response to Reviewer TqEb (Part-1)

Thank you for highlighting how the policy-energy bottleneck connects assessment to targeted intervention and for recognizing the strong detection and rewriting results. We address your questions and concerns point by point below.

W1 & Q: Isolating effects of different framework parts (Part-1)

Thank you for identifying the controlled comparisons needed to isolate the contribution of geometric policy memory. PolicyMem is designed as a learnable, external interface for policy knowledge, rather than a standalone safety detector. Natural-language policies are compiled into individually addressable geometric slots, and their policy-indexed evidence is reused across assessment, attribution, targeted rewriting, and post-rewrite verification. To examine what this shared interface contributes, we conduct five complementary controls covering backbone adaptation, verdict capacity, trained per-policy predictors, rewriting feedback, and direct attribution-source replacement. Most importantly, replacing PolicyMem's policy attribution with a trained policy-text reader, while keeping the rewriter and evaluation fixed, lowers the joint good-rewrite rate from 0.7677 to 0.7253. The experiments below establish how the representation affects both intermediate policy evidence and the downstream correction it guides.

(i) Representation versus backbone adaptation. We freeze the backbone LoRA adapter and train only the governance-specific modules: the case projection, subspace compiler, and soft-token projection. The adapter remains active during the forward pass but receives no parameter updates; policy anchors follow the same detached refresh procedure. This reduces trainable parameters from 12.74M to 3.94M, or approximately 31% of the reference setup.

Table 1R: Policy memory with the backbone adapter frozen.

MetricPooled-5 referencePooled-5 memory onlyPooled-4 referencePooled-4 memory only
Trainable parameters12.74M3.94M12.74M3.94M
Policy attribution Hit@1.6959.6038.7751.6835
Mean off-diagonal subspace overlap.1879.1984.1952.1995

With backbone adaptation held fixed, the governance modules retain 86.8% and 88.2% of the respective reference attribution Hit@1 using only 31% of the trainable parameters. The learned slots also maintain similar inter-policy separation. This demonstrates that governance-specific modules can learn discriminative, individually addressable policy evidence without further updating the backbone adapter, supporting the value of an explicit policy representation beyond backbone adaptation alone.

(ii) Verdict-only controls with specified supervision and capacity. We clarify the anchor-based detector in Appendix C.7 and evaluate additional hidden_only alternatives. These alternatives predict the safety verdict from the pooled case representation instead of the policy-energy summary. They use verdict cross-entropy without the geometric alignment objectives and do not expose policy-indexed scores at the decision interface. We vary their additional decision capacity while keeping the data and training schedule fixed.

Table 2R: Verdict-pathway capacity control (capacity-study protocol, 0.5B backbone).

ArmAdditional decision parametersIn-distribution macroHeld-out WildGuardMix
PolicyMem, policy-energy bottleneck0.8688.7170
Verdict-only control, extra_dim = 6489,154.8642.7227
Verdict-only control, extra_dim = 10241,441,794.8748.7207

Even with up to 1.44M additional verdict-specific parameters, the alternatives offer no consistent advantage across in-distribution and held-out evaluation. PolicyMem remains within 0.6 percentage points of the strongest alternative on either metric, without introducing a separate verdict MLP. More importantly, its decisions arise from an explicit policy-evidence profile that is directly reusable for attribution, targeted rewriting, and policy-specific reassessment. A verdict-only pathway can reassess whether a rewrite is safe, but does not provide the indexed residual-policy evidence needed for targeted follow-up intervention. The parameter counts above are additional decision parameters, not total trainable governance parameters.

(iii) Comparably trained per-policy predictors. We compare against a conventional P-way sigmoid multi-label head trained on the same frozen case representation z, and PECOS XR-Linear [1], a published extreme multi-label ranker. The comparisons use matched new-policy labeling budgets and the same evaluation cases. PECOS is evaluated across four configurations, combining frozen z or TF-IDF features with flat or hierarchical label organization; we report the strongest configuration at each budget.

AuthorsOpenReview ↗

Response to Reviewer TqEb (Part-2)

W1 & Q: Isolating effects of different framework parts (Part-2)

Table 3R: New-policy attribution mAP at matched new-policy labeling budgets.

Labeled examples per new policy251030
P-way multi-label head.1297.2290.3042.4550
PECOS XR-Linear.1551.2289.3054.5410
PolicyMem.3320.4708.5285.6688

PolicyMem outperforms both trained predictors at every tested new-policy labeling budget, with the strongest relative advantage in the few-shot regime. At only five labeled examples per policy, PolicyMem already reaches .4708 mAP, exceeding both alternatives at ten examples per policy. Moreover, description-based slot insertion requires no new-policy labels or gradient updates: in the 38-to-58-policy expansion, a newly compiled slot reaches .2517 mAP before alignment, compared with .2290 for the P-way head trained with five examples per policy.

Stability under policy expansion. PolicyMem also preserves the energies and relative rankings of existing cached policy slots when new slots are inserted without learning updates. In the corresponding incumbent-policy evaluation, the top-1 attribution changes on 0.0% of cases, compared with 8.9% to 12.2% for the P-way head and 2.5% to 42.1% across PECOS configurations. This stability follows directly from individually addressable reads: adding a policy does not require recomputing the evidence attached to an existing slot. Together, the acquisition and maintenance results highlight the practical value of text-grounded policy initialization and stable, reusable policy evidence.

(iv) Matched policy-conditioned rewriting controls. We hold the same trained rewriter checkpoint, the fixed evaluation cohort, greedy decoding, and both independent judges (o3 and GPT-5.2) constant. Only the policy feedback supplied to the rewriter changes.

Table 4R: Rewriting-conditioning ladder (Eq. 23 joint good-rewrite rate).

Rewriter feedbackJoint good-rewrite ratePrompt tokens
Generic safety instruction.7395361 (0.88x)
All 77 policy descriptions.73451,906 (4.67x)
All policies from the case's taxonomy.7568653 (1.60x)
PolicyMem Top-3 attributed policies.7677408 (1.00x)

Focused PolicyMem feedback achieves the highest joint good-rewrite rate using only 408 prompt tokens. Compared with providing all policies, it improves joint success by 3.32 percentage points while using 4.67 times fewer tokens. It also outperforms generic safety instructions, despite the latter having a shorter prompt. These controls show that selecting the relevant policy knowledge matters more than simply adding policy text or increasing prompt length.

(v) Direct attribution-source replacement in the single-pass intervention pipeline. To test whether improved attribution affects the final correction rather than only an intermediate metric, we replace PolicyMem's Top-3 with the Top-3 supplied by a trained, block-isolated policy-text reader. The trained rewriter, evaluation cohort, greedy decoding, and independent judges remain unchanged.

Table 5R: Attribution-source replacement in the rewriting pipeline.

Attribution sourceEq. 23 joint good-rewrite rate
Trained, block-isolated policy-text reader.7253
PolicyMem Top-3.7677

PolicyMem improves joint rewriting success by 4.24 percentage points, with higher scores on all five benchmarks. The feedback diagnostics explain this downstream difference: PolicyMem's Top-3 contains a gold policy in 96.0% of cases, versus 88.6% for the trained text reader, and the two Top-3 sets overlap by only 1.66 of 3 policies on average. This is the direct evidence connecting PolicyMem's representation-level advantage to more effective corrective rewriting.

Additional task-adapted guard baselines. Our original evaluation follows established cross-benchmark safeguard practice [2,3,4], using released, safety-specialized checkpoints with their official inference interfaces. To complement that deployment-oriented comparison, we additionally fine-tune Granite Guardian, WildGuard, and MD-Judge on a shared adaptation sample, using the same LoRA recipe and official output formats. They achieve .8741 / .8684 / .8411 macro Unsafe-F1, respectively, while PolicyMem (3B specialist) reaches .8874 with 2.3 to 2.7 times fewer backbone parameters. These results establish strong detection performance against both released and additionally adapted guards, while the controlled experiments above isolate the operational contribution of the policy-memory interface.

The revised manuscript includes the controlled feedback and geometric/onboarding analyses in Appendices C.5, C.10, and C.11, highlighting how shared, addressable policy evidence improves downstream corrective intervention.

AuthorsOpenReview ↗

Response to Reviewer TqEb (Part-3)

W2: Independent safety and utility evaluation of the full loop

Thank you for emphasizing that closed-loop governance must be evaluated through independent delivery safety and legitimate usefulness, not only the assessor's own stopping decisions. The paper's main rewriting comparison (Figure 3 and Appendix B.3.3) deliberately uses one rewrite attempt per method, assessed by independent judges; the bounded rewrite-verification loop is analyzed separately in Appendix C.4. We now complement these experiments with independent evaluation of the final delivered outputs and an end-to-end utility assessment on a separate corpus.

Independent judgment of complete-loop deliveries. We evaluate the same fixed case cohort under single-pass and bounded-retry delivery, holding inputs, the trained rewriter, decoding, and judging rubric constant; only the verification-guided retry procedure changes. Neither o3 nor GPT-5.2 participates in training or internal stopping decisions. The independent safety rate is 99.17% after one rewrite and 99.04% after the bounded loop, compared with the assessor's 99.86% accepted fraction. Both independent safety rates exceed 99% and are within one percentage point of the assessor's result. Thus, the delivered-response safety is corroborated outside the training-and-verification loop.

Human annotation of delivered outputs. We additionally evaluate a stratified sample of 100 final delivered responses, comprising all 7 cases scored below the safety threshold by either independent judge, 13 borderline cases, and 80 randomly sampled cases. Three human annotators, blinded to the judges' scores and the sampling strata, assessed 54, 53, and 53 assigned cases, respectively, yielding safe-label rates of 94.4%, 94.3%, and 96.2%. Among the 93 cases outside the low-scoring group, agreement with the independent-judge safety threshold ranges from 97.8% to 100%. This human evaluation provides complementary evidence of delivered-response safety, with disagreements concentrated in cases already identified as challenging by the independent judges.

Final delivery and legitimate usefulness on an external benchmark. We further evaluate the deployed governor end-to-end on XSTest [5], separate from the five safety benchmarks. The 200 genuinely unsafe prompts probe harmful delivery; the 250 adversarially benign prompts probe unnecessary intervention, over-refusal, and task completion.

Table 6R: XSTest end-to-end evaluation under independent judgment.

MetricResult
Harmful delivery on 200 unsafe prompts.0000
Over-refusal on 250 adversarially benign prompts, governor off to on.1280 to .1280
Task completion on the same 250 prompts, governor off to on.8200 to .8200
False-flag rate on adversarially benign prompts0.4%

No harmful deliveries are observed on the evaluated unsafe subset. On the adversarially benign prompts, PolicyMem introduces no measured increase in over-refusal or reduction in task completion: the rates remain 12.8% and 82.0%, respectively, and only 0.4% of benign responses are flagged. Combined with the independent assessment of full-loop deliveries, these results support safe final responses while preserving legitimate-task utility in the evaluated settings, without relying solely on the assessor's own verification signal.

References:

[1] PECOS: Prediction for Enormous and Correlated Output Spaces. JMLR

[2] WildGuard: Open One-stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs. NeurIPS 2024

[3] BingoGuard: LLM Content Moderation Tools with Risk Levels. ICLR 2025

[4] GuardReasoner: Towards Reasoning-Based LLM Safeguards. ICLR 2025 Workshop

[5] Xstest: A test suite for identifying exaggerated safety behaviours in large language models. NAACL 2024

Thank you again for the constructive review. We welcome discussion if you have any further questions.

Dear Reviewer TqEb,

Thank you again for recognizing the value of our unified policy representation and for your constructive suggestions. Following your comments, we have further strengthened our experimental validation through controlled comparisons of policy representations and rewriting strategies, along with independent evaluations of end-to-end safety and utility.

As we approach the end of the discussion phase, we would like to ask whether our rebuttal has addressed your concerns. We would be happy to provide any further clarifications if needed.

Thank you again for your time and consideration!

Best regards,
Authors

Reviewer_dPj4OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces PolicyMem, a framework for LLM governance that represents natural-language safety policies as learned low-rank geometric subspaces. These representations are used across unsafe-response detection, policy attribution, response rewriting, and post-rewrite verification. The system employs a shared policy assessor that measures the alignment between query-response representations and policy subspaces. The resulting policy evidence guides a separately trained rewriter and is reused to verify revised responses. Experiments across five safety benchmarks demonstrate strong detection and policy-attribution performance, together with promising rewriting quality and sequential policy onboarding.

Critical strengths
  1. Interesting unified policy representation: The proposed geometric policy memory provides a coherent mechanism for connecting safety detection, policy attribution, and corrective rewriting. The use of individually addressable policy representations is potentially valuable for policy management and adaptation.

  2. Strong experimental results: The evaluation covers five safety benchmarks and includes comparisons with multiple established guard models. The reported results are generally strong, particularly considering the relatively small model sizes. The additional ablation, efficiency, and policy-onboarding experiments strengthen the experimental contribution.

Critical weaknesses
  1. Limited evidence isolating the methodological novelty: Although the geometric policy representation is interesting, the paper combines several established techniques, including learned embeddings, low-rank projections, contrastive learning, and iterative rewriting. The reported ablation against an anchor-based MLP is useful but insufficient to establish the necessity of the proposed subspace formulation. Comparisons with simpler learned policy embeddings, similarity-based classifiers, and other capacity-matched alternatives would help demonstrate the specific contribution of the geometric architecture.

  2. Insufficiently controlled baseline comparisons: PolicyMem is explicitly trained on the corresponding benchmark datasets, whereas many of the evaluated guard models are used as pretrained checkpoints without equivalent task-specific fine-tuning. Similarly, the policy-attribution comparison mainly includes prompted general-purpose LLMs rather than comparably trained attribution models. These differences make it difficult to determine how much of the improvement comes from the proposed architecture rather than task-specific supervision. Stronger comparisons under matched training data and computational budgets are needed.

  3. Limited independent validation of the closed-loop framework: The proposed rewriter is trained using feedback from the PolicyMem assessor, which is also used for subsequent verification. Although the paper evaluates single-pass rewriting using independent LLM judges, the reported convergence of the complete rewrite-verification loop relies on the same assessor that supplies the training signal. Therefore, a successful verification does not necessarily establish that the revised response is independently safe or policy-compliant. Evaluating the complete loop using independent judges or human annotations would strengthen the conclusions.

Questions for authors
  1. Can the authors provide comparisons with simpler, capacity-matched policy representation methods trained using the same data and objectives? In particular, what is the specific benefit of representing each policy as a low-rank subspace rather than a learned policy embedding?
  2. How would PolicyMem compare with existing guard models or conventional multi-label classifiers when all approaches receive equivalent task-specific training data and computational budgets?
  3. Can the authors evaluate the complete multi-round rewriting and verification pipeline using independent safety judgments rather than relying on the same assessor used to train the rewriter?
  4. How well does the proposed approach generalize to more complex organizational policies, including overlapping requirements, exceptions, conflicting policies, and updates to existing policies?
6 comments
AuthorsOpenReview ↗

Response to Reviewer dPj4 (Part-1)

We thank the reviewer for recognizing the value of a unified, individually addressable geometric policy memory and the strong results across five safety benchmarks. Here are our point-by-point responses to your questions.

W1 & Q1: Comparison with simpler representation methods (Part-1)

Thank you for encouraging us to isolate the contribution of geometric policy memory. Our central contribution is an external, learnable interface for policy knowledge. Policies are compiled into individually addressable geometric slots, and the same policy-level evidence is reused for response assessment, violation attribution, targeted rewriting, and post-rewrite verification. This design connects the stages of closed-loop governance through a shared policy representation rather than stage-specific policy processing. We evaluate both the representation itself and its consequences for intervention. Most notably, replacing PolicyMem's attribution with that of a trained policy-text reader, while holding the rewriter and evaluation fixed, reduces the joint good-rewrite rate from 0.7677 to 0.7253. The following controlled comparisons explain where this downstream advantage originates.

(i) Learned policy embeddings. With a unit-norm case representation, a rank-1 policy slot under our squared-projection read is equivalent to a learned policy vector scored by squared cosine similarity. We therefore vary only the subspace rank, keeping the training data, objectives, and optimization recipe fixed.

Table 1R: Effect of policy-subspace rank.

RankMean pairwise slot overlapAttribution mAP
1 (learned policy embedding).0817.7481
4.0758.7641
8 (deployed setting).0667.7631
16.0742.7696

Compared with rank 1, the deployed rank-8 memory reduces mean pairwise slot overlap by 18.4% (.0817 to .0667) and improves attribution mAP (.7481 to .7631), with comparable detection (.8961 versus .8951). Rank 16 further increases mAP to .7696. These results show that multi-directional policy subspaces produce more discriminative, better-separated policy evidence than the rank-1 control under matched training, without relying on a larger verdict head. Importantly, this is the same indexed evidence subsequently used to guide rewriting.

(ii) Embedding, similarity, and entailment alternatives. We additionally compare against dedicated embedding models, cosine similarity over the chat backbone, and a DeBERTa-v3 natural-language-inference (NLI) analyzer. NLI tests policy entailment rather than embedding similarity; its result is the stronger of two fixed templates evaluated under the same policy set and attribution cases.

Table 2R: Semantic baselines for attribution. All read-outs rank the co-trained governor's union of 77 configured policies (Appendix B.3.2; Figure 7), a harder candidate set than the per-benchmark setting of Table 2.

Read-outTop-1Top-3mAP
Qwen3-Embedding-0.6B.152.368.256
Qwen3-Embedding-4B.142.401.269
all-mpnet-base-v2.167.393.261
DeBERTa-v3 NLI.149.330.261
Chat-backbone cosine (mean pooling / last token).062 / .074.200 / .244.153 / .167
PolicyMem.753.918.768

PolicyMem achieves .768 mAP and .918 Top-3 recall, nearly 3 times the mAP of the strongest training-free alternative (.269).

(iii) Capacity-controlled verdict alternatives. To distinguish policy evidence from decision-head capacity, we also evaluate verdict-only pathways that bypass the policy-energy summary, with additional decision capacities of 89,154 and 1,441,794 parameters (extra_dim = 64 and 1024). Under the same capacity-study training schedule, their in-distribution macro scores are .8642 / .8748, compared with .8688 for the 0.5B PolicyMem reference run of the same sweep; held-out WildGuardMix scores are .7227 / .7207, compared with .7170. Thus, even a 16-fold increase in the alternative pathway's width changes detection by at most 0.6 percentage points relative to PolicyMem on either evaluation, without providing a consistent capacity-dependent improvement. PolicyMem remains competitive without a separate verdict MLP, using its permutation-invariant policy-evidence summary and the backbone's existing LM head. More importantly, unlike a verdict-only pathway, the indexed evidence remains directly available for attribution, targeted rewriting, and reassessment.

AuthorsOpenReview ↗

Response to Reviewer dPj4 (Part-2)

W1 & Q1: Comparison with simpler representation methods (Part-2)

(iv) Text-grounded policy compilation. We hold the geometry, rank, policies, subsequent replay/alignment procedure, and evaluation cases fixed, changing only whether a new policy slot is initialized from its natural-language description or a random anchor.

Table 3R: Text-compiled versus randomly initialized policy anchors.

InitializationText-compiled anchorRandom anchor
New-taxonomy Unsafe-F1.8701.2361
New-policy attribution Top-1 / mAP.7208 / .7730.1193 / .2660
Incumbent Unsafe-F1.8711.8696

Text-grounded initialization improves new-taxonomy Unsafe-F1 by 63.4 percentage points and new-policy mAP from .2660 to .7730, while maintaining incumbent performance. Because subsequent alignment is identical, this comparison isolates the value of the description-grounded initialization rather than an extra training advantage. Together with the no-update onboarding results in W2, it shows that a new policy description provides an informative memory slot immediately and a substantially stronger starting point for subsequent alignment.

Downstream impact: replacing the attribution source. Finally, we replace PolicyMem's Top-3 attribution with that of a trained, block-isolated policy-text reader in the same single-pass detection-attribution-rewriting evaluation. The trained rewriter, evaluation cases, greedy decoding, and both independent judges remain fixed. The Eq. 23 joint good-rewrite rate decreases from .7677 to .7253, with PolicyMem ahead on all five benchmarks. PolicyMem's Top-3 contains a gold policy in 96.0% of cases, versus 88.6% for the text reader, and the feedback sets overlap on only 1.66 of 3 policies on average. This is the key end-to-end finding: more accurate, individually addressable policy evidence leads to measurably better corrective rewriting, beyond a standalone improvement in detection.

W2 & Q2: Controlled guard-model and trained-attribution comparisons (Part-1)

Thank you for raising the question of supervision and baseline comparability. Our original comparisons use released, safety-specialized guard checkpoints with their official inference formats, consistent with established cross-benchmark safeguard evaluation practice [1-3]. These are meaningful deployment baselines because they already embody substantial safety-oriented training; they are not unadapted general-purpose LLMs. To strengthen the architectural comparison, we now provide additional task-adapted guard baselines and trained policy-attribution and multi-label controls. We additionally fine-tune Granite Guardian, WildGuard, and MD-Judge on a shared adaptation sample, using the same LoRA recipe across the three guards. Evaluation uses the same five test splits and each model's recommended inference interface.

Table 4R: Released and task-adapted guard results.

ModelParametersReleased checkpointFine-tuned
Granite Guardian8B.8356.8741
WildGuard7B.8633.8684
MD-Judge7B.8576.8411
PolicyMem0.5BN/A.8727
PolicyMem3BN/A.8874

The strongest task-adapted guard reaches .8741, while the 3B PolicyMem achieves .8874 with 2.3 to 2.7 times fewer backbone parameters. The 0.5B PolicyMem also exceeds every released checkpoint and the additionally fine-tuned WildGuard at roughly one-fourteenth the size of a 7B guard. These experiments extend our original results beyond released checkpoints, while the representation-controlled comparisons provide complementary evidence for the contribution of geometric policy memory beyond detector capacity.

Trained policy attribution under controlled conditions. We compare the policy memory with two trained LLM-based text readers using the same backbone family, LoRA configuration, training collection, policy labels, and step budget. The results below are measured under a pooled policy evaluation protocol.

Table 5R: Trained attribution controls on the pooled policy evaluation.

Read-outUnsafe-F1Attribution mAPTop-1 changes under policy-list reordering
Policy-text list reader.888.71928.3%
Block-isolated per-policy reader.889.7925.5%
PolicyMem.894.7970.0%

All three methods yield competitive detection, but PolicyMem achieves the highest attribution mAP while preserving exact invariance of retained policy energies to list ordering. For the list reader, changing only the order of the same policies changes the top-ranked attribution on 28.3% of cases. A cached PolicyMem slot, by contrast, is read independently of other slots' positions. This gives the downstream rewriter a more stable policy-level signal.

AuthorsOpenReview ↗

Response to Reviewer dPj4 (Part-3)

W2 & Q2: Controlled guard-model and trained-attribution comparisons (Part-2)

Matched k-shot multi-label prediction. We next compare with a conventional P-way sigmoid head on the same frozen case representation z and PECOS XR-Linear [4], a published extreme multi-label ranker. The P-way head and PECOS use the same incoming-policy label budgets; PECOS is evaluated with frozen z or TF-IDF features and flat or hierarchical label structures. We report PolicyMem under the corresponding k-shot onboarding budgets rather than comparing a full alignment result against few-shot baselines.

Table 6R: New-policy attribution mAP by labeled examples per incoming policy.

Methodk=2k=5k=10k=30
P-way multi-label head.1297.2290.3042.4550
PECOS XR-Linear.1551.2289.3054.5410
PolicyMem.3320.4708.5285.6688

PolicyMem leads both trained predictors at every tested label budget. The advantage is particularly relevant in low-data onboarding: at k = 5, PolicyMem reaches .4708 mAP, exceeding the P-way head and PECOS even at k = 10 (.3042 and .3054). With no target-policy labels or gradient update, compiling the incoming policy description alone already yields .2517 mAP in the 38-to-58-policy expansion, versus .2290 for the P-way head at k = 5. This demonstrates the immediate value of a policy description as an addressable memory entry, followed by efficient refinement when labeled cases become available.

Stability under policy expansion. A second benefit is the separation between structural insertion and parameter updates. In insertion-only mode, cached PolicyMem slots preserve incumbent per-policy energies exactly, resulting in 0.0% changes to incumbent-only Top-1 attribution. The evaluated P-way heads change 8.9% / 12.2% of incumbent Top-1 predictions at k = 5 / 30, while PECOS configurations yield 2.5% to 42.1% changes after their respective updates. These are different update mechanisms, but they expose an important operational distinction: PolicyMem can add a description-grounded slot without first modifying the representations of existing policies. Under the reported onboarding updates, retained-policy mAP also remains close, from .7914 at k = 0 to .7782 at k = 30.

Implications for closed-loop governance. The comparisons identify three complementary benefits of a shared external policy-knowledge interface. Effectiveness: replacing PolicyMem's attribution with the trained text reader reduces single-pass joint good-rewrite rate from .7677 to .7253. Stable, addressable reads: cached slots preserve incumbent energies under insertion and are invariant to policy-list order. Efficiency: when the configured policy set grows from 77 to 277, the block-isolated text reader's latency rises from 69.8 to 240.9 ms per case, versus 24.9 to 25.7 ms for PolicyMem (H100, batch 8). The gain is therefore not confined to a detection score. The same compact policy representation provides actionable evidence throughout the governance loop without reprocessing the entire policy corpus for each case.

W3 & Q3: Independent evaluation of the complete rewrite-verification loop (Part-1)

Thank you for identifying the importance of evaluating the complete loop independently of the assessor that supplies the rewriter's training signal. The paper's main rewriting comparison (Figure 3; Appendix B.3.3) deliberately uses one rewrite per method, judged externally, so that baseline comparisons have the same intervention budget. The bounded rewrite-verification loop is analyzed separately in Appendix C.4. We now evaluate delivered outputs from the complete loop using independent LLM judges, further validate their safety through human annotation, and test end-to-end safety and benign-task utility on a disjoint corpus.

(i) Independent judges of complete-loop outputs. On the same evaluation cohort, we hold the trained rewriter, case set, decoding, and external judges fixed, varying only whether verification-guided retry is enabled. Neither o3 nor GPT-5.2 is used for rewriter training or internal stopping.

Table 7R: Delivered-response safety under independent judgment.

SettingAssessor-safe fractionIndependently judged safe fraction (score ≥ 3)
One rewrite, no retry.9986.9917
Bounded multi-round retry.9986.9904

Independent judges rate 99.17% of single-pass deliveries and 99.04% of complete-loop deliveries as safe, compared with the assessor's 99.86% acceptance rate. Both independent estimates are within one percentage point of the assessor's rate, directly addressing the concern that safety is established solely by self-verification. For the fully trained rewriter, first-pass safety is already high, leaving little room for further improvement from retries.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.