acceptodds
Under review as a conference paper at ICLR 2027

You Got the Right Answer for the Wrong Reason: Process Fairness in Agentic AI via Structured Deliberation

Abstract

Tool-using language model agents can treat matched applicants differently in their decisions, explanations, and evidence-gathering trajectories. We formalize these channels and introduce PRISM (Pre-Registered Inference with Stereotype Masked Scaffolding), a black-box protocol that fixes a case-independent rubric, threshold, fallback policy, and required tools before demographic exposure, then audits case-level adherence. On the evaluated DiscrimEval-derived and Legal/GRC audits, PRISM reduces paired outcome divergence by 52–62% and 82–84%, respectively, replicates across four backbones including a 30B agent-first model, and lowers reasoning-trace divergence on both open-world profiles. Matched controls suggest that committing to criteria before identity exposure, rather than who writes them, carries the effect. With executed tools, PRISM also raises required-tool coverage from 76% to 99.8% and removes tool-set mismatch between matched applicants, and an independent re-run reproduces 99% of its decisions.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
agentic fairness, process fairness, LLM agents, counterfactual fairness, structured deliberation, demographic bias

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ajDuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

As AI systems are increasingly deployed as "agents," able to engage in multi-step reasoning, whether these intermediate choices are fair becomes all the more important. The authors develop an auditing framework of multiple steps of agents to assess their fairness. They then develop a scaffold to improve fairness which they call PRISM.

Critical strengths

I think the idea of evaluating the fairness of intermediate agentic choices is very clever and important. I think the problem the authors take on (and the initial framing in the intro) is very good, and I learned something there!

I also generally think the approach the authors take with PRISM in separating criteria formation and case eval is sensible, and I appreciate that the authors engage in both eval and productive scaffold design.

Critical weaknesses

Unfortunately, I really struggled to understand the heart of the paper and the results. I found Section 3 and 4 admittedly quite confusing. I'm not sure if this was from AI writing (though I suspect that was a substantial part), or my naivete (also possible, and I've flagged lower confidence from myself in the review!)

I do appreciate the authors had key takeaways! Though some like "evidence boundary" are confusing and not quite paper-publishing-ready, in my opinion. For instance, that one says "supported" (but it's not clear the specific hypothesis?)

I think the body of those sections would benefit from a more substantive rewriting.

The appendix is also extremely hard to follow. Again, I think this is substantively AI-generated.... which seems to lead to quite confusing and unclear structuring. At minimum, adding a table of contents would help...

Questions for authors

I loved the idea of assessing the fairness of tool use. It could be worth bringing some qualitative exploration of tools used into the main text? (say for track B) that could help with clarity and grounding the work for readers!

Reviewer_nQsvOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces the methodology called PRISM for auditing LLMs on legal-based tasks.

Critical strengths

The project is timely and important to study. The idea of counterfactual fairness in LLMs is quite interesting and has a lot of interesting downstream consequences.

Critical weaknesses
  1. The paper is quite overloaded and difficult to follow. I really tried to understand the methodology but I was unable to do so. For more concrete questions, please see the box below.
  2. Reproducibility. There is no codebase included.
Questions for authors
  1. Can you please explain the methodology in two to three sentences?
  2. What kind of tools is your model using -- how are these defined and how is it called?
Reviewer_rmDfOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors focus on demographic disparities in tool-using language-model agents. They propose PRISM, a two-stage protocol that fixes a case-independent tools before demographic exposure, then evaluates the applicant using that commitment. Experiments use paired demographic variants in a controlled DiscrimEval-derived setting and a Legal/GRC setting with freely selected tools. The findings include: (1) PRISM reduces paired decision mismatch across several backbones and improves mandatory-tool coverage; (2) process metrics do not improve uniformly, and (3) policy-label accuracy decreases in the primary Qwen setting.

Critical strengths
  1. The paper introduce a practical intervention with replicated outcome-consistency gains.
  2. The reported results are solid: (1 )mismatch reductions occur across two audit settings and multiple backbones, and (2) the scenario-clustered intervals support improvements over the baseline.
  3. The additional controls materially improve the scientific value. These controls, together with explicit reporting of worsened process metrics and reduced policy accuracy, make the study informative even when they limit the proposed explanation.
  4. In particular, the fixed-rubric result suggests that a stable precommitted policy may be more important than model-generated criteria.
Critical weaknesses
  1. My first concern is that the process-fairness constructs are insufficiently validated. BERTScore differences may also reflect formatting wording, and tool-count or depth differences need not measure unjustified scrutiny. Conversely, equal omission counts can conceal different omitted tools. The paper acknowledges these limitations, but they directly affect its process-fairness framing: the procedural composite worsens by approximately 18–23% in the primary open-world experiments. Without independent judgments, the evidence supports improvements in selected procedural-consistency measures rather than a general improvement in process fairness.

  2. My second concern is the distinctive benefit of the full protocol remains uncertain relative to simpler controls and its utility cost. (1) The timing contrast in Appendix is suggestive, but its confidence interval includes 0. (2) The fixed author-written rubric essentially matches PRISM. On Qwen’s financial profile, policy-label accuracy falls from 0.930 to 0.866. The external threshold rule obtains zero paired disparity with accuracy 0.859, close to PRISM’s. These findings weaken the case that model-generated pre-registration is necessary or that the full protocol actually offered the best fairness–utility trade-off.

Questions for authors
  1. Can a blinded, independently assessed subset establish whether changes in reasoning and trajectory metrics correspond to changes in unjustified scrutiny?

  2. What advantage does PRISM retain over the fixed-rubric and externally enforced threshold controls when compared jointly on policy accuracy, paired disparity, tool cost, and latency?

  3. How do mismatch reductions compare with identical-input noise under both Baseline and PRISM?

2 comments
Area_Chair_SgevOpenReview ↗

Please check and acknowledge the author response to your review

Dear reviewer — as the discussion phase is coming to a close, please acknowledge and respond to the author responses to your review. It would be helpful to provide time for the authors to comeback with further follow up if necessary.

Thanks AC

Initially, I felt this topic had merit, but after reading the vague response provided during the rebuttal, I find it difficult to follow how my concerns are being addressed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.