acceptodds
Withdrawn from ICLR 2027

MASCRDM: Multi-Agent System for Compliance Risk Detection and Mitigation in Training Process of Large Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large Language Models (LLMs) have been applied in various fields. However, ensuring compliance and safety of LLMs, such as avoiding discrimination and bias, still remains a challenge. Current efforts mainly focus on detecting and filtering inputs and outputs of the trained models, rather than studying the intrinsic architecture of the models in real-time. To tackle this challenge, we analyze the LLMs training process and discover two critical issues: 1) Most of the existing methods are predominantly static in their approach to detection and filtering, achieving only localized optimizations without systematically enhancing the compliance of LLMs. 2) Another issue with existing approaches is the lack of real-time risk detection and mitigation across the full training process, which leads to limited flexibility. Motivated by these, we propose MASCRDM (Multi-Agent System for Compliance Risk Detection and Mitigation) during the LLM training process. Firstly, we develop a set of compliance rules based on existing Artificial Intelligence (AI) laws and a compliance-specific LLM with the instruction of compliance law experts. Then, we deconstruct LLMs into several components and identify key nodes based on the compliance knowledge graph. During LLMs training, we implement our multiple agents in the whole process, giving compliance risk alerts and suggestions for LLM developers. Experiments on discrimination and bias benchmark demonstrate that our multi-agent system can effectively improve the compliance while maintaining reasonable semantic performance. The results indicate that our method provides an executable path for mitigating compliance risk from within the LLMs systematically.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Compliance, LLM Training, Risk Detection, Risk Mitigation, Multi-Agent System

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_qpK1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents MASCRDM, a framework for monitoring and mitigating compliance risks during LLM training. An AI Compliance Knowledge Graph (AICKG) connects regulatory requirements to risks, training stages, and intervention rules. Multiple agents coordinate interventions in data, embeddings, attention, MLPs, and the training objective. Experiments on Qwen3-8B and Llama-3.1-8B evaluate bias mitigation and capability preservation.

Critical strengths
  1. The motivation to connect governance requirements with actionable training-time interventions is important. Moving beyond output filtering toward monitoring the training process could enable developers to identify and address problems earlier.

  2. Experiments on two model families and component ablations provide useful evidence of competitive utility-bias trade-offs.

Critical weaknesses
  1. Meaning of the risk signals. Demographic-versus-neutral representation differences need not indicate unjustified discrimination. A concrete trace from a regulatory requirement to a justified alert would clarify what legal grounding adds to ordinary fairness probes.

  2. Detection validity. The experiments mainly measure final bias and accuracy. These improvements support mitigation, but do not establish whether training-time alerts identify genuinely problematic behavior. Agreement with reference judgments or downstream discriminatory behavior would directly support the detection claim.

  3. Coordination benefit and cost. Component ablations do not isolate AICKG scheduling or multiple agents from the underlying interventions. A fixed pipeline or single controller using identical modules would clarify their practical benefit, particularly given the reported 55.75% training-time overhead.

Questions for authors
  1. Can a regulatory requirement be traced to an alert validated against discriminatory behavior?
  2. What independent validation is available for the training-time detectors?
  3. What benefit does the coordination framework provide over identical interventions with simpler scheduling?
Reviewer_mXpZOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents MASCRDM, a training-time framework for detecting and mitigating bias and discrimination risks in language models. It introduces an AI Compliance Knowledge Graph that links regulations, risk categories, training stages, and detection rules. Five named agents coordinate planning, data auditing, representation-level detection, objective-level detection, and mitigation. The implementation audits training samples, monitors embedding completion gaps, attention distributions, MLP activations, and loss-level preferences on 36 neutral-anchored triplets, and applies sample reweighting or rewriting, embedding correction, attention projection, MLP gradient gating, and a neutral-margin objective. The method is evaluated after fine-tuning Qwen3-8B and Llama-3.1-8B on Alpaca-Cleaned augmented with StereoSet examples. Results are reported on BBQ, CrowS-Pairs, and BOLD against three debiasing baselines.

Critical strengths
  1. The paper studies bias mitigation throughout model training instead of adding only an output filter. It considers data, embeddings, attention, MLP activations, and the training objective within one pipeline. The neutral-anchored probes also avoid treating the mere presence of a protected attribute as evidence of bias.

  2. The supplementary package substantially improves inspectability. It includes the 53,883-example source corpus, the offline-audited version with retained original outputs and sample weights, the 36 triplet probes, modular training code, fixed hyperparameters, and reproducibility checks. The paper also reports training overhead and includes an extensive component table, so readers can examine how different interventions affect ambiguous and disambiguated BBQ results.

Critical weaknesses
  1. The paper does not validate the claimed connection between legal compliance and the implemented bias signals. The AICKG maps regulations to risks and training stages, but the experiments evaluate only social bias on BBQ, CrowS-Pairs, and BOLD. There is no evaluation of rule retrieval, legal mapping accuracy, alert precision or recall, root-cause attribution, or whether a mitigation actually satisfies the regulation that triggered it. The paper also does not report the number or qualifications of the legal experts or agreement on the graph. The implemented signals are standard bias proxies based on paired prompts and internal differences. Calling them compliance-risk measurements requires evidence that the AICKG changes a technical decision and that this decision matches the cited requirement.

  2. The end-to-end gains are modest and inconsistent across metrics. On Qwen3-8B, MASCRDM reaches 63.12 BBQ accuracy, only 0.11 points above Bias Unlearning, while its CrowS-Pairs score of 63.00 remains far from the neutral value of 50 and is worse than Bias Unlearning's 61.74. Many BOLD values are identical to the base model or competing methods at the reported precision. No repeated training runs, confidence intervals, or significance tests are provided. The full-system ablation is also not consistently better than simpler variants: several partial configurations exceed its 63.12 BBQ accuracy, including Data Audit + MLP + Loss at 63.82, while Data Audit alone reaches 63.41. These results do not establish that all four representation and objective modules, or the multi-agent organization, are necessary.

  3. The role of real-time agents is unclear in the actual experiment. The supplementary README states that MASCRDM trains on an offline-audited dataset in which 1,034 examples have already been rewritten and 323 have been downweighted. It is not clear whether the Data Risk Detection Agent audits these samples again during training, which data version is used by each baseline, or how much of the final gain comes from offline preprocessing. The Planning Agent also appears to retrieve fixed rules, while most other agents implement deterministic statistics and update rules. The paper should distinguish the decisions made by an LLM agent from ordinary monitoring modules and compare the proposed organization with the same interventions executed by a fixed training loop.

  4. The supplementary material does not fully reproduce the paper's complete claim. It provides a final modular implementation for Qwen3-8B and materialized offline audit decisions, but not the compliance-specific LLM, the full AICKG, expert mapping records, the process that generated the rewritten samples, Llama-3.1-8B runs, trained checkpoints, or baseline outputs. Therefore, the supplied package supports inspection of one training implementation but cannot verify the legal-grounding pipeline or the complete experimental comparison. The reported 55.75% training-time overhead is also substantial for a system whose advantage over simpler ablations is small.

Questions for authors

See Weaknesses.

Reviewer_v66HOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

MASCRDM uses a compliance knowledge graph and five agents to detect and reduce bias-related risks during LLM fine-tuning. It combines data auditing, interventions in model representations, and a margin loss on 36 probe triplets. On two 8B models, it reports the highest overall BBQ accuracy among the compared full methods, alongside results on CrowS-Pairs and BOLD.

Critical strengths
  • Connecting regulatory requirements to checks during training is a useful direction, beyond filtering model inputs or outputs.
  • Several detector choices make sense, including shared neutral references, probe-wise maxima to retain localized signals, and layer-specific MLP thresholds.
  • The experiments cover two models, three benchmarks, and component ablations. On Llama-3.1-8B, disambiguated accuracy is substantially better than Bias Unlearning: 50.14 versus 41.31.
Critical weaknesses
  1. The two “critical issues” in the Abstract are asserted without a training-process analysis showing them. Describing earlier methods as “single-model paradigms” also does not explain why multiple agents are needed. Separately, the margin loss favors the neutral reference over both stereotypical and counter-stereotypical probes (Appendix D.5.4). Whether this affects appropriate group-specific statements outside the probes is not examined.

  2. Detection is a central claim, but the experiments do not measure detection quality. There are no detection accuracy or false-positive results, counts of flagged or rewritten samples, or example alerts. Better post-training bias scores do not tell us whether the detectors identify risks correctly or provide useful feedback to developers.

  3. I could not assess some key components from the description. The compliance-specific LLM's base model, training data, and evaluation are missing. The knowledge graph has node examples and legal sources, but its construction and validation need more detail. In particular, how legal requirements are translated into detector criteria needs justification; listing the source laws alone does not validate that mapping.

  4. It is unclear what agent coordination adds beyond the individual components. A pipeline with the same detectors and interventions, but without agent coordination, would be a useful control. In Table 5, data auditing alone reaches 63.41 accuracy versus 63.12 for the full system; 13 of 15 ablated variants also have higher accuracy. The full system has a better B.Amb score than data auditing alone (3.32 vs. 3.76), but without results across seeds, it is hard to judge this trade-off.

  5. Some gains are small and their source is unclear. On Qwen3-8B, the margins over Bias Unlearning are 0.11 in accuracy and 0.05 in B.Amb, without uncertainty estimates. CrowS-Pairs ranks third on both models, and BOLD is reported only for Qwen, with mostly small differences among methods other than KLAAD. Also, the probe loss applies regardless of whether a risk is detected (Appendix D.5.4). A loss-only baseline is needed to separate this effect from detection-triggered mitigation. The probe contents should be disclosed so their relationship to the evaluation items can be checked.

Questions for authors
  1. How often do the detectors fire, and how accurate are their decisions on independently labeled examples?
  2. How was the compliance LLM trained and evaluated, and how were the knowledge graph's legal-to-detector mappings validated?
  3. How were the 36 probes constructed, and what results do you get with their margin loss alone?
  4. What do the base rows in Tables 1 and 5 represent, and why do they differ?
  5. Across seeds, does the full system offer a reliable accuracy–bias improvement over data auditing alone or the same components without agent coordination?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.