- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors attempt to examine the fairness of language models within the embedding space. However, the current experimental evaluation falls far short of adequately addressing this topic. The manuscript requires substantial improvement, and I do not believe these shortcomings can be resolved during the discussion phase.
Critical strengths
None
Critical weaknesses
1. Unclear Dataset Construction and Preprocessing
- The procedure for converting the tabular Adult dataset into template sentences lacks sufficient documentation. The paper mentions dropping several "complex numerical" attributes during this conversion, yet fails to specify which attributes were removed or the rationale behind their exclusion.
- Basic dataset statistics and splitting protocols are omitted. The authors do not report the total dataset size, the train, validation, and test split ratios, the splitting methodology, or the procedure used to balance classes.
- For the toxicity experiments, the text refers to a "curated version of the Jigsaw dataset" without describing the curation criteria or filtering steps.
2. Unreported Pretraining Data Composition and Unexamined Embedding Quality
- The composition of the embedding pretraining data is not documented. There is no mention of the number of gendered words, the counts and ratios of positive (attracted) and negative (repelled) word pairs, or the specific source and definition of the "broad English vocabulary" used to sample negative pairs.
- The resulting pretrained embeddings exhibit clear quality flaws whose downstream effects remain unanalyzed. Unrelated tokens appear within the gender clusters. In Table 14, the 50 nearest neighbors to "female" and "male" include terms such as "deer", "bull", "lion", "busch", and "gonzales", which bear no semantic relation to gender or appear for reasons that cannot be explained. Since these 50 neighbors directly define the verification threshold, these unrelated tokens are effectively treated as gendered words during verification.
- Evidence for successful pretraining relies solely on a small set of hand-picked word pairs (Tables 15, 17, and 18) alongside a subjective statement that manual inspection showed mostly related terms, lacking any quantitative analysis across the full vocabulary. In particular, the closest token to "bitch" is
[PAD], and its nearest neighbors include general words ("shy", "pack", "pick", "mother") as well as punctuation symbols (;, ,+,(,!, ). - Coverage of task-relevant vocabulary during pretraining is completely unreported. Since the embeddings remain frozen during classifier training, any token absent from the pretraining pairs retains its random initialization when fed into the classifier. Task-relevant tokens in the Adult sentences, such as occupation titles and marital status, heavily influence classification outcomes, yet the paper never reports what fraction of these tokens appeared in the pretraining pairs.
3. Ambiguity in Data Augmentation
- According to the Appendix, the authors applied WordNet synonym replacement for data augmentation, which alone increased classification accuracy from 80% to approximately 90%.
- However, the augmented sentences contain grammatically broken and semantically distorted phrases such as "sex be male", "human relationship is Husband", and "aborigine country".
- Furthermore, the paper fails to specify which experimental configurations in Table 1 used this augmentation. Given that synonym replacement alone shifts accuracy by roughly 10 percentage points, the accuracy comparisons in Table 1 cannot be meaningfully interpreted without knowing where augmentation was applied.
4. Unspecified Hyperparameter Tuning Protocol and Potential Test Leakage
- The authors state that hyperparameters were adjusted empirically to achieve high accuracy and a large verification radius, which falls short of a rigorous scientific protocol.
- No details are provided regarding what data or criteria guided this adjustment. If hyperparameters were selected by inspecting verification outcomes on the 50 test sentences, the reported fairness scores suffer from test leakage and are artificially inflated. The authors must clarify whether a separate validation set was held out for hyperparameter selection.
5. Disconnect Between the Evaluated Classifier and Standard Language Models
- The evaluated model does not represent a standard language model. It is a tiny Transformer classifier trained from scratch under supervision, configured with an 8-dimensional embedding, 2 attention heads, and an 8-dimensional feed-forward layer. It was never trained with a language modeling objective, nor does it use pretrained language model weights.
- By evaluating an arbitrarily small, custom-trained classifier without verifying whether it is properly trained, practically useful, or representative of deployed language models, the experimental setup lacks generalizability and connection to prior literature.
- Essential architectural details are missing, including positional encodings, normalization layers, classification head structure, and
[PAD]token handling. Since formal verification results depend directly on the exact network architecture, these omissions prevent both reproduction and proper interpretation. - All reported results come from a single training run, with no mean or standard deviation reported across multiple random seeds.
- Dropout rates vary across the compared models, leaving key experimental variables improperly controlled.
- Although the experiments are restricted to a tiny custom classifier, the title, abstract, and introduction motivate the work through large language models without providing evidence that the findings transfer to actual language models. In addition, the related work section states that the paper focuses on generated text, whereas no text generation experiments are conducted. This represents an overgeneralization and a substantial logical leap. If scaling the method to realistic language models is computationally intractable, the authors must establish a rigorous theoretical or empirical link between their custom classifier and real-world language models.
6. Circularity in the Verification Criterion
- In formal verification, the target model and the verification specification must be defined independently. Standard practice first establishes an external specification, such as a fixed list of gendered word pairs ("male" and "female", "man" and "woman", "he" and "she") under which predictions must remain invariant, and subsequently tests whether the model satisfies that fixed list.
- Instead, the authors couple the verification criterion directly to the model under test. They extract the 50 nearest neighbors to "male" and "female" from the model's own embedding space and set the distance to the 50th neighbor (
3e-04) as the verification radius. - This setup is analogous to allowing an examinee to select the 50 questions they know best for a test while excluding harder words, which undermines the practical meaning of the verification guarantee.
- As seen in Table 14, the 50 nearest neighbors of "female" include unrelated words such as "deer", "bull", "lion", "busch", and "gonzales". Consequently, the certificate merely guarantees that replacing "female" with "deer" leaves the prediction unchanged, without guaranteeing invariance for genuinely gendered words that fall outside those 50 neighbors.
- The choice of 50 neighbors is also arbitrary. As the authors acknowledge, increasing the neighbor count enlarges the distance threshold (
3e-04for 50 neighbors,6e-04for 150 neighbors, and larger for 200). As a result, the exact same model changes from "fair" to "unfair" simply by adjusting the neighbor count from 50 to 200. This metric therefore reflects an arbitrary parameter chosen by the evaluator rather than an intrinsic property of the model.
7. Discrepancy Between Methodology and Experiments
- Algorithm 1 defines a sentence-specific verification criterion determined by synonym distances across all words appearing in each individual sentence.
- In contrast, the experimental section applies a single uniform gender-word threshold across all test sentences.
8. Insufficient Sample Size and Missing Baselines
- Fairness verification is evaluated on only 50 sentences, which is far too small to establish statistical significance. Meanwhile, classification accuracy is evaluated on a test set of unreported size.
- No baseline methods are included in the experiments. A simple baseline that replaces all gendered words with a single shared token would trivially achieve complete invariance to gender substitutions. The authors need to demonstrate what advantages their method offers over such a straightforward alternative.
- Although the related work section surveys multiple existing fairness verification methods, none of them are empirically compared against the proposed approach.
9. Undefined Fairness Notion and Incomplete Ablation Studies
- The paper never defines whether the targeted fairness notion corresponds to individual fairness, group fairness, or counterfactual fairness, and it provides no empirical evaluation to verify actual fairness behavior. Even the most direct empirical check, measuring the prediction flip rate when explicitly substituting "male" with "female" in test sentences, is absent.
- In Section 4, the authors claim that training the embeddings jointly with the classifier prevents gendered words from clustering together, yet they never evaluate initializing the model with the pretrained embeddings and subsequently fine-tuning them jointly.
- The individual contributions of each pretraining stage are not isolated. There is no ablation study separating the effect of gender-word pretraining from general synonym pretraining.
- Finally, the paper provides no sensitivity analysis examining how verification scores vary with key pretraining hyperparameters, such as the attraction strength, margin values, and the composition of the target word lists.
Questions for authors
see weakness