acceptodds
Under review as a conference paper at ICLR 2027

GROVE: Governed Rule Operationalization through Verifiable Agentic Evolution for Multimodal Moderation

Abstract

How can a multimodal moderator adapt to an authorized policy revision without retraining its vision–language model or silently changing unrelated rules? We introduce GROVE (Governed Rule Operationalization through Verifiable Agentic Evolution), a two-stage framework that compiles policy clauses, exceptions, and scopes into a versioned schema of atomic questions. Its release protocol specifies source-bound agent edits and independent checks of policy fidelity and protected behavior. At inference, five-choice answer-token distributions supply rule-linked evidence to a lightweight decision adapter. We evaluate this evidence path on five public image–text benchmarks with group-isolated predictions and fixed-readout controls. Adding atomic evidence to a fixed XGBoost head improves macro-F1 by 3.93 points on MAMI, ROC-AUC by 1.56 points on Hateful Memes, and reject F1 by 8.69 points on HARM-C relative to the same head using only the base score. A policy-free representation head serves as a demanding control. The measured contribution is an informative, traceable evidence channel; sequential policy-update evaluation is specified separately.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Content Moderation, Verifiable Policy Updates, Lightweight Decision Adaptation, Vision Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_NZWuOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces GROVE, a framework for operationalizing policy changes in multimodal content moderation without retraining the underlying vision–language model. It decomposes an authorized Policy Book into source-linked atomic questions with explicit answer semantics, applicability conditions, and version information. An agent proposes constrained schema edits, while an independent verifier controls their release. At inference time, a lightweight adapter combines five-choice answer distributions with a base moderation score to support correction or human review

Critical strengths
  1. The problem formulation is practical and clearly motivated.
  2. The separation of normative authority from operational schema editing is a strong design choice.
Critical weaknesses
  1. The central “agentic evolution + verifier-gated release” contribution is not actually evaluated.
  2. The improvement from atomic evidence does not by itself establish improved policy reasoning; it may partly reflect additional feature engineering and additional VLM computation.
  3. The comparison with full-policy prompting and other policy-adaptive guardrails is insufficient.
  4. The computational cost is not fully characterized. In particular, the reported historical latency can be substantial, and compute-matched comparisons are missing.
Questions for authors
  1. Can you provide completed policy-update experiments covering changes to prohibitions, exceptions, and scope, with results for authorized decision flips, invariant-case preservation, and verifier rejection outcomes?
  2. What information from the evaluation cohorts was used during catalog development? Were labels or error analyses inspected, and can you provide an independent evaluation after freezing the catalogs?
1 comment

We thank the reviewer for the detailed and constructive review.

Q2. Information used during catalog development; independent evaluation

The fixed catalogs were developed on the benchmark cohorts before the reported readout audit (Appendix E). Outer-fold isolation applies to readout fitting, calibration and threshold selection. The current manuscript does not record which labels, predictions or error analyses were consulted at the catalog stage. We agree that this is a real concern, and we treat the original results as retrospective evidence.

New independent evaluation. We froze the original catalog, both readouts and their configurations, and then scored a new cohort of 600 cases (240 positives, one case per independent content group) that was never used in development:

Same head, fixed policyROC-AUC
Base score only84.69%
Base score + atomic evidence85.92%
Paired difference (95% CI)+1.23 pp [0.84, 1.64]

The gain from atomic evidence holds on data that played no part in catalog design.

Q1 / W1. Completed policy-update experiment and verifier outcomes

We completed an update that adds an exception: neutral, attributed quotation or reporting becomes exempt, while endorsed quotations and separate unexempted attacks are still rejected. Explicit rebuttal was already exempt under the old policy. Gold labels are assigned independently under both policy versions. The final audit is held out from both editing and sealed verification.

Sealed verifier gate. The frozen candidate passes all prespecified gates:

CheckResultPrespecified gate
Correct reject→allow flips49/60 (one-sided 95% lower bound 71.5%)≥45/60 and lower bound >60%
Endorsed quotations wrongly allowed0/600/60
Separate attacks wrongly allowed0/600/60
New errors or deferrals, previously correct invariant cases7/284≤5%
New errors or deferrals, previously correct protected cases0/1120
Deferrals (sealed set / protected set)10/360; 2/120≤10% on each

All 60 content groups in the protected set contain at least one case that was correctly classified under the old policy.

Final audit (separate data). GROVE makes 47/60 correct flips, has 7/300 unintended flips and 8/286 new errors or deferrals, and wrongly allows 0/60 endorsed quotations and 0/60 separate attacks. Full-policy prompting scores 44/60, 14/300, 22/285, 4/60 and 5/60.

Verifier rejection. The clause-semantics check rejects an injected edit that makes quotation marks sufficient for an exemption. On a separate fault-test set with that check bypassed, the edit wrongly allows 9/60 endorsed quotations.

Scope. The completed experiment covers one exception. We have not yet run experiments for changes to prohibitions or to scope, and we will state this as a limitation in the revision. The verification protocol itself (source, scope and protected-case checks) does not depend on the type of change.

W2 / W3. Source of the gains; full-policy prompting and other guardrails

Table 1 holds the readout fixed and measures the value of the whole atomic feature package: HARM-C FP fall from 720 to 431 and FN from 251 to 238. To separate the effect of clause decomposition from that of extra computation, we compared batched clause-linked questions against direct full-policy prompting. Both use the same VLM and inputs, one call per case and common token caps. On the 360-case new-policy audit, GROVE has 10/180 FP, 1/180 FN and 10/360 deferrals. Full-policy prompting has 16/180, 11/180 and 14/360. Because both arms have nearly the same compute, the gain cannot come mainly from extra VLM computation. SingGuard and PolicyShiftGuard are relevant policy-adaptive guardrails. We will discuss them in the related work. A direct comparison with them remains future work.

W4. Computational cost

  • Matched setting (new). Both methods use one call per case per policy. Mean token use is 1,530 for GROVE vs. 1,480 for full-policy prompting under the new policy, and 1,450 vs. 1,410 under the old policy. Both differences are within 10%. End-to-end latency p50 / p95 is 2.1 / 4.0 s for GROVE vs. 1.7 / 3.6 s.
  • Historical setting (Table 7). The original serving configuration made seven separate calls per case: a median of 5,241 total tokens and 35.0 s summed API latency per Hateful Memes case, and a median of 401.1 s on HARM-C. These numbers exclude retries. Batching the questions into a single call removes most of this overhead. We will report both settings in the revision.
Reviewer_HVNTOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes GROVE, a framework for adapting multimodal content moderation systems to policy changes without retraining the underlying vision-language model. The proposed framework consists of two stages: a policy operationalization stage that converts policy clauses into atomic questions and supports verifier-gated agentic evolution, and an inference stage that extracts atomic evidence distributions and combines them with a lightweight decision adapter. The experiments evaluate the predictive utility of atomic evidence on five multimodal moderation benchmarks. The authors report improvements over score-only baselines, particularly on MAMI, Hateful Memes, and HARM-C. Additional experiments compare the policy-evidence branch against representation-based models and investigate disagreement-based human review.

Critical strengths
  1. An interpretable and modular design for policy-based moderation.

The proposed framework explicitly separates policy definitions, atomic evidence extraction, and final moderation decisions. This decomposition provides a potentially useful interface for inspecting and updating policy-related decisions without modifying the underlying VLM.

  1. Positive empirical results under controlled readout comparisons.

The experiments demonstrate that incorporating atomic evidence improves a fixed XGBoost classifier over a score-only baseline on several benchmarks. The use of group-isolated evaluation and paired statistical analysis provides useful evidence for the predictive value of the additional atomic features.

Critical weaknesses
  1. The empirical advantages of atomic policy evidence remain insufficiently established against strong baselines and under realistic evaluation conditions.
  2. The central contribution of agentic policy evolution and verifier-gated release is not experimentally validated.
  3. The writing lacks logical clarity.
Questions for authors
  1. Could the author please provide a clear description of the problem background, the limitations of existing methods, the motivation, and the proposed solution?

  2. I would kindly suggest that the author further refine their work.

2 comments

We thank the reviewer for the feedback. Below we restate the problem and the contribution, and we report the new experiments.

Q1. Background, limitations of existing methods, motivation and solution

  • Problem. Moderation policies change often. After an update, a system should change the decisions that the new rule affects and keep all other decisions unchanged. For example, after adding a neutral-quotation exception, a meme that reports an attack should become allowed, while a meme that endorses the same attack must still be rejected.
  • Limitations of existing methods. Fine-tuned classifiers must be retrained after each policy change. Policy-conditioned VLMs can read the revised policy text, but they give no trace of how the change was implemented and no test for regressions before deployment. In our experiments, full-policy prompting changes 14/300 decisions that should not change.
  • Solution (GROVE). (1) Policy clauses become atomic questions, each linked to the clause that authorizes it. (2) An agent proposes versioned edits to these questions when the policy changes. (3) A separate verifier checks each edit's source, scope and effect on protected cases, and only the verifier can release it. (4) A lightweight adapter combines the answer distributions with a base score. The VLM stays frozen throughout.

W1. Strong baselines and realistic evaluation

  • Fixed-policy results: with the same XGBoost configuration, atomic features raise HARM-C reject-class F1 from 61.70% to 70.39% (Table 1). The policy-free image–text model has higher point estimates on four of five tasks (Table 2). On Hateful Memes, however, the policy-evidence branch reaches 89.58% ROC-AUC vs. 88.99%, a paired difference of +0.59 pp (95% CI [0.09, 1.13]). We do not claim that GROVE is the most accurate fixed-policy classifier. Its purpose is to support policy updates that can be traced and checked for regressions.
  • New: matched-budget comparison with full-policy prompting (same VLM, one call per case, common token caps, 360 cases). GROVE has 10/180 FP, 1/180 FN and 10/360 deferrals. Full-policy prompting has 16/180, 11/180 and 14/360. The cost is +3.4% tokens and +0.4 s latency.
  • New: independent evaluation with frozen catalogs (600 cases, one case per content group). Atomic evidence improves ROC-AUC from 84.69% to 85.92% (+1.23 pp, 95% CI [0.84, 1.64]).

W2. End-to-end validation of the agentic evolution and the verifier

We ran a complete update for the quotation exception: authorized change → agent edit → sealed offline verification → separate 360-case final audit. On the final audit, GROVE makes 47/60 correct flips and 7/300 unintended flips, with 8/286 new errors or deferrals among previously correct invariant cases. It wrongly allows 0/60 endorsed quotations and 0/60 separate attacks. Full-policy prompting scores 44/60, 14/300, 22/285, 4/60 and 5/60. The verifier rejected an injected edit that treats any quotation as exempt; with the check bypassed, that edit wrongly allows 9/60 endorsed quotations. Full details are in our reply to Reviewer DqTr.

W3 / Q2. Writing clarity

We will restructure the paper as follows. The introduction will be built around the quotation example. The method section will follow the order problem → design → verification. A short paragraph will separate what the fixed-policy tables establish (the predictive value of atomic evidence) from what the policy-update experiment evaluates (correct flips, invariance and verifier gating).

Thank you for your response. Having reviewed the questions raised by myself and the other reviewers, along with your replies, I have decided to keep my score unchanged.

Reviewer_DqTrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GROVE addresses a practical problem: when a moderation policy changes, how can the system update the decisions affected by the new rule without changing unrelated decisions? The method turns policy clauses and exceptions into questions for a vision–language model without retraining it. An agent proposes edits to those questions, and a separate verifier checks whether the edited version can be released. The paper reports results under fixed policies on five public benchmarks.

Critical strengths
  1. Each proposed edit can be traced back to the policy clause that authorizes it. This makes it possible to check whether an agent has implemented a new rule or changed the policy itself.
  2. The fixed-policy experiment is useful on its own. In Table 1, adding features from the policy questions raises reject-class F1 on HARM-C from 61.70% to 70.39% with the same XGBoost configuration. Both false positives and false negatives decrease.
Critical weaknesses
  1. The paper does not show a policy change carried through from edit to release. Section 5.3 and Appendix D describe the experiment but report no completed run. Take the quotation example: after adding the exception, does GROVE stop rejecting posts that quote an attack while still rejecting posts that endorse it? Results on cases held out from editing, along with an example of a bad edit caught by the verifier, would directly test the main claim.

  2. It is unclear whether breaking the policy into questions is worth the extra inference. Table 1 compares the added features against a single base score. In Table 2, the image–text baseline without policy features has higher point estimates on four of five tasks. How does GROVE compare with giving the same model the full policy directly under the same inference budget?

Questions for authors
  1. Can you show one complete policy update, such as adding a quotation exception? On cases held out from editing, how many decisions that should change actually change in the right direction, and how many decisions that should stay the same change by mistake? Did the verifier catch any proposed edits that caused errors?
  2. With the same vision–language model and matched call and token budgets, how do rule-linked questions compare with direct prompting using the full policy in false positives, false negatives, and latency?
1 comment

We thank the reviewer for the precise questions. Both requested experiments are now complete.

Q1 / W1. A complete policy update: the quotation exception

Setup. Under the old policy, only explicit rebuttal is exempt. The authorized update adds an exemption for clearly attributed, neutral quotation or reporting. A meme that endorses the quoted attack, or that contains a separate unexempted attack, must still be rejected. Gold labels are assigned to the same image and text under both policy versions. Related memes are kept together when we split development, sealed verification and final audit data, so no content group crosses splits. The final audit (360 cases) is never used for editing or verification. Of these 360 cases, 60 should flip from reject to allow and 300 should keep their labels.

Metrics. A correct flip requires both predictions (old policy and new policy) to match their gold labels. New errors or deferrals are counted only over invariant cases that the method classified correctly under the old policy, so the denominator differs slightly by method. The no-update control reuses the old-policy decisions.

Final audit (held out from editing and verification):

MethodCorrect flips ↑Unintended flips ↓New errors / deferrals ↓Endorsed quotes wrongly allowed ↓Separate attacks wrongly allowed ↓
No update0/600/3000/2861/601/60
Full-policy prompting44/6014/30022/2854/605/60
GROVE47/607/3008/2860/600/60
  • Correct flips: GROVE − full-policy = +5.0 pp (paired 95% CI [−10.0, 20.0]). The two methods are comparable on applying the new exception.
  • New errors or deferrals on previously correct invariant cases: −4.9 pp (paired 95% CI [−8.09, −1.88]). The main advantage of GROVE is that it leaves out-of-scope decisions unchanged, which is the property the reviewer asked about.
  • GROVE does not over-extend the exception. It wrongly allows no endorsed quotation and no separate attack.

Verifier outcomes. The frozen candidate passes the sealed offline gate with 49/60 correct flips. Bad-edit example: we injected a candidate edit that exempts any attack in quotation marks, and the clause-semantics check rejected it. In a separate fault test with that check bypassed, the bad edit wrongly allows 9/60 endorsed quotations. The verifier therefore blocks a concrete failure mode before release. The final audit and the fault test use different case sets.

Q2 / W2. Matched-budget comparison with full-policy prompting

Both arms use the same VLM, the same image, visible text and OCR, one inference call per case per policy, and common input and output token caps. GROVE batches the applicable atomic questions into this single call. The comparator receives the complete active policy. Results on the 360-case new-policy audit:

MethodFP (of 180 gold-allow) ↓FN (of 180 gold-reject) ↓Deferred (of 360) ↓Calls / caseMean tokensLatency p50 / p95 (s)
Full-policy prompting16111411,4801.7 / 3.6
GROVE1011011,5302.1 / 4.0

GROVE uses 3.4% more tokens under the new policy (1,530 vs. 1,480). Under the old policy, the counts are 1,450 vs. 1,410. Both ratios are within our prespecified 10% tolerance. GROVE adds 0.4 s at both p50 and p95. Under a matched budget, clause-linked questions therefore reduce both error types at a modest latency cost. On Table 2: the representation baseline is a policy-free fixed-policy classifier, so it cannot follow a policy change without retraining. The comparison above measures the policy-conditioned alternative that the reviewer asked about.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.