- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates how demographic information is encoded in language models using linear probes trained on blog posts. It examines 29 base models across six families, tracks OLMo 2 training checkpoints, and studies how decodability changes with model size and context length. It also evaluates the transfer of blog-trained gender probes to real conversations and uses activation interventions to examine their role in explicit gender prediction.
Critical strengths
-
The evaluation across multiple model families, scales, and training stages provides a systematic empirical account of demographic information in model representations.
-
Transfer to three real conversation datasets strengthens the practical relevance of the findings.
-
Author-disjoint splits, author-clustered bootstrapping, and lexical, embedding-layer, and random-initialization controls improve the reliability of the evaluation.
Critical weaknesses
-
Mean-pooled hidden representations may reflect demographic attributes, writing style, and topic simultaneously. The existing lexical controls do not fully distinguish these factors. Additional controls or a narrower interpretation of “user attribute representations” would strengthen the conclusions.
-
Label mapping and user filtering are not fully specified for each dataset. The paper should clarify how label categories and missing values are handled and report user counts before and after filtering.
-
The abstract makes parallel claims about gender and age, whereas analyses of training trajectories and conversational transfer primarily concern gender. Corresponding age analyses should be provided, or the claims should be qualified accordingly.
-
The robustness of the shared scaling slopes requires further validation. Model families cover different size ranges, and Pythia shows a different trend for gender prediction. Confidence intervals for the slopes and a comparison between shared-slope and family-specific-slope models would strengthen this conclusion.
Questions for authors
-
How are user histories ordered and concatenated in the real-conversation experiments? How are overlength inputs truncated, and what token budget is used for each model?
-
When an AITA post contains age and gender markers for multiple people, how is the poster’s own label identified? How are ambiguous cases handled?
-
Appendix A.1 states that the training set was not deduplicated. Have the authors checked whether the main findings hold after removing near-duplicates and texts not written by the recorded author from the training set?
4.Have the authors considered jointly predicting gender and age, reporting the proportion of cases where both are correct and performance across the six combinations of gender and age group? Has stratified evaluation been conducted to assess the influence of correlations between these attributes?