acceptodds
Under review as a conference paper at ICLR 2027

Language Models That Walk the Talk: A Framework for Formal Fairness Certificates

Abstract

As large language models become integral to high-stakes applications, ensuring their robustness and fairness is crucial. Despite their success, large language models remain vulnerable to adversarial attacks, where small perturbations can alter model predictions. In language models, perturbations such as synonym substitutions also pose risks in fairness- and safety-critical areas. Formal verification is a viable path to certify fairness; however, its application to large language models remains limited. This work presents a holistic verification framework to certify the fairness of language models by formalizing the verification problem within the embedding space, with a focus on ensuring gender fairness and consistent outputs across different gender-related terms. Furthermore, we extend this methodology to toxicity detection to offer formal guarantees that adversarially manipulated toxic inputs are consistently detected and appropriately censored, thereby ensuring the reliability of moderation systems.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
ai, safety, fairness, formal methods

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 2 reviews, above 0% of the others.
Reviewer_7iAqOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors attempt to examine the fairness of language models within the embedding space. However, the current experimental evaluation falls far short of adequately addressing this topic. The manuscript requires substantial improvement, and I do not believe these shortcomings can be resolved during the discussion phase.

Critical strengths

None

Critical weaknesses

1. Unclear Dataset Construction and Preprocessing

  • The procedure for converting the tabular Adult dataset into template sentences lacks sufficient documentation. The paper mentions dropping several "complex numerical" attributes during this conversion, yet fails to specify which attributes were removed or the rationale behind their exclusion.
  • Basic dataset statistics and splitting protocols are omitted. The authors do not report the total dataset size, the train, validation, and test split ratios, the splitting methodology, or the procedure used to balance classes.
  • For the toxicity experiments, the text refers to a "curated version of the Jigsaw dataset" without describing the curation criteria or filtering steps.

2. Unreported Pretraining Data Composition and Unexamined Embedding Quality

  • The composition of the embedding pretraining data is not documented. There is no mention of the number of gendered words, the counts and ratios of positive (attracted) and negative (repelled) word pairs, or the specific source and definition of the "broad English vocabulary" used to sample negative pairs.
  • The resulting pretrained embeddings exhibit clear quality flaws whose downstream effects remain unanalyzed. Unrelated tokens appear within the gender clusters. In Table 14, the 50 nearest neighbors to "female" and "male" include terms such as "deer", "bull", "lion", "busch", and "gonzales", which bear no semantic relation to gender or appear for reasons that cannot be explained. Since these 50 neighbors directly define the verification threshold, these unrelated tokens are effectively treated as gendered words during verification.
  • Evidence for successful pretraining relies solely on a small set of hand-picked word pairs (Tables 15, 17, and 18) alongside a subjective statement that manual inspection showed mostly related terms, lacking any quantitative analysis across the full vocabulary. In particular, the closest token to "bitch" is [PAD], and its nearest neighbors include general words ("shy", "pack", "pick", "mother") as well as punctuation symbols (;, , +, (, !, ).
  • Coverage of task-relevant vocabulary during pretraining is completely unreported. Since the embeddings remain frozen during classifier training, any token absent from the pretraining pairs retains its random initialization when fed into the classifier. Task-relevant tokens in the Adult sentences, such as occupation titles and marital status, heavily influence classification outcomes, yet the paper never reports what fraction of these tokens appeared in the pretraining pairs.

3. Ambiguity in Data Augmentation

  • According to the Appendix, the authors applied WordNet synonym replacement for data augmentation, which alone increased classification accuracy from 80% to approximately 90%.
  • However, the augmented sentences contain grammatically broken and semantically distorted phrases such as "sex be male", "human relationship is Husband", and "aborigine country".
  • Furthermore, the paper fails to specify which experimental configurations in Table 1 used this augmentation. Given that synonym replacement alone shifts accuracy by roughly 10 percentage points, the accuracy comparisons in Table 1 cannot be meaningfully interpreted without knowing where augmentation was applied.

4. Unspecified Hyperparameter Tuning Protocol and Potential Test Leakage

  • The authors state that hyperparameters were adjusted empirically to achieve high accuracy and a large verification radius, which falls short of a rigorous scientific protocol.
  • No details are provided regarding what data or criteria guided this adjustment. If hyperparameters were selected by inspecting verification outcomes on the 50 test sentences, the reported fairness scores suffer from test leakage and are artificially inflated. The authors must clarify whether a separate validation set was held out for hyperparameter selection.

5. Disconnect Between the Evaluated Classifier and Standard Language Models

  • The evaluated model does not represent a standard language model. It is a tiny Transformer classifier trained from scratch under supervision, configured with an 8-dimensional embedding, 2 attention heads, and an 8-dimensional feed-forward layer. It was never trained with a language modeling objective, nor does it use pretrained language model weights.
  • By evaluating an arbitrarily small, custom-trained classifier without verifying whether it is properly trained, practically useful, or representative of deployed language models, the experimental setup lacks generalizability and connection to prior literature.
  • Essential architectural details are missing, including positional encodings, normalization layers, classification head structure, and [PAD] token handling. Since formal verification results depend directly on the exact network architecture, these omissions prevent both reproduction and proper interpretation.
  • All reported results come from a single training run, with no mean or standard deviation reported across multiple random seeds.
  • Dropout rates vary across the compared models, leaving key experimental variables improperly controlled.
  • Although the experiments are restricted to a tiny custom classifier, the title, abstract, and introduction motivate the work through large language models without providing evidence that the findings transfer to actual language models. In addition, the related work section states that the paper focuses on generated text, whereas no text generation experiments are conducted. This represents an overgeneralization and a substantial logical leap. If scaling the method to realistic language models is computationally intractable, the authors must establish a rigorous theoretical or empirical link between their custom classifier and real-world language models.

6. Circularity in the Verification Criterion

  • In formal verification, the target model and the verification specification must be defined independently. Standard practice first establishes an external specification, such as a fixed list of gendered word pairs ("male" and "female", "man" and "woman", "he" and "she") under which predictions must remain invariant, and subsequently tests whether the model satisfies that fixed list.
  • Instead, the authors couple the verification criterion directly to the model under test. They extract the 50 nearest neighbors to "male" and "female" from the model's own embedding space and set the distance to the 50th neighbor (3e-04) as the verification radius.
  • This setup is analogous to allowing an examinee to select the 50 questions they know best for a test while excluding harder words, which undermines the practical meaning of the verification guarantee.
  • As seen in Table 14, the 50 nearest neighbors of "female" include unrelated words such as "deer", "bull", "lion", "busch", and "gonzales". Consequently, the certificate merely guarantees that replacing "female" with "deer" leaves the prediction unchanged, without guaranteeing invariance for genuinely gendered words that fall outside those 50 neighbors.
  • The choice of 50 neighbors is also arbitrary. As the authors acknowledge, increasing the neighbor count enlarges the distance threshold (3e-04 for 50 neighbors, 6e-04 for 150 neighbors, and larger for 200). As a result, the exact same model changes from "fair" to "unfair" simply by adjusting the neighbor count from 50 to 200. This metric therefore reflects an arbitrary parameter chosen by the evaluator rather than an intrinsic property of the model.

7. Discrepancy Between Methodology and Experiments

  • Algorithm 1 defines a sentence-specific verification criterion determined by synonym distances across all words appearing in each individual sentence.
  • In contrast, the experimental section applies a single uniform gender-word threshold across all test sentences.

8. Insufficient Sample Size and Missing Baselines

  • Fairness verification is evaluated on only 50 sentences, which is far too small to establish statistical significance. Meanwhile, classification accuracy is evaluated on a test set of unreported size.
  • No baseline methods are included in the experiments. A simple baseline that replaces all gendered words with a single shared token would trivially achieve complete invariance to gender substitutions. The authors need to demonstrate what advantages their method offers over such a straightforward alternative.
  • Although the related work section surveys multiple existing fairness verification methods, none of them are empirically compared against the proposed approach.

9. Undefined Fairness Notion and Incomplete Ablation Studies

  • The paper never defines whether the targeted fairness notion corresponds to individual fairness, group fairness, or counterfactual fairness, and it provides no empirical evaluation to verify actual fairness behavior. Even the most direct empirical check, measuring the prediction flip rate when explicitly substituting "male" with "female" in test sentences, is absent.
  • In Section 4, the authors claim that training the embeddings jointly with the classifier prevents gendered words from clustering together, yet they never evaluate initializing the model with the pretrained embeddings and subsequently fine-tuning them jointly.
  • The individual contributions of each pretraining stage are not isolated. There is no ablation study separating the effect of gender-word pretraining from general synonym pretraining.
  • Finally, the paper provides no sensitivity analysis examining how verification scores vary with key pretraining hyperparameters, such as the attraction strength, margin values, and the composition of the target word lists.
Questions for authors

see weakness

Reviewer_RaouOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Large language models (LLMs) suffer from vulnerable to adversarial attacks, where small perturbations may change models' predictions. This paper presents a holistic verification framework to certify the fairness of LLMs by formalizing the verification problem within the embedding space. This work focuses on ensuring gender fairness and consistent outputs across different gender-related terms. The same methodology is further extended to toxicity detection.

In my opinion, putting selected word substitutions inside an embedding-space neighborhood, and then trying to certify that the classifier’s output stays fixed across that whole neighborhood is a good idea. However, the paper does not support these claims of formal fairness for LLMs. My major concerns are whether the transformer verification is fully sound, whether the proposed “fairness” score measures fairness, and whether the experiments test a broad enough model or input set.

Critical strengths
  1. The paper is well-organized. It covers related works and limitations.

  2. Putting selected word substitutions inside an embedding-space neighborhood, and then trying to certify that the classifier’s output stays fixed across that whole neighborhood is a good idea.

Critical weaknesses
  1. The certificate is not well-examined. The appendix introduces zonotope propagation through affine layers and nonlinear activations, but it does not show how the verifier can handle the operations like attention dot products, softmax, and layer normalization. Only when every operation is enclosed soundly, the certificate is valid. The authors should explain more on the implementation with more details and validate it against exact enumeration on small cases.

  2. The so-called "fairness score" in this paper is a specific token-level counterfactual invariance property. It does not establish group fairness, nor does failure to certify prove unfairness. The paper should explain how it relates to established fairness definitions, and report measures such as demographic-parity or equal-opportunity gaps alongside it.

  3. This paper does not compare any related works in the result section. The authors should consider comparing their techniques against the following works: (1) Gregory Bonaert, Dimitar I Dimitrov, Maximilian Baader, and Martin Vechev. Fast and precise certification of transformers. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, pp. 466–481, 2021. (2) Yibin Wang, Yichen Yang, Di He, and Kun He. 2023. Robustness-Aware Word Embedding Improves Certified Robustness to Adversarial Word Substitutions. In Findings of the Association for Computational Linguistics: ACL 2023, pages 673–687

Questions for authors

Please comment on the points raised in the weakness section. Particularly, the point 3: can the authors compare this work against the following works: (1) Gregory Bonaert, Dimitar I Dimitrov, Maximilian Baader, and Martin Vechev. Fast and precise certification of transformers. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, pp. 466–481, 2021. (2) Yibin Wang, Yichen Yang, Di He, and Kun He. 2023. Robustness-Aware Word Embedding Improves Certified Robustness to Adversarial Word Substitutions. In Findings of the Association for Computational Linguistics: ACL 2023, pages 673–687 in the result section?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.