- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
In this work, the authors argue and show that unfairness can arise from an ensemble of fair prediction agents. They begin with a toy example illustrating such a result in theory before introducing notation and theorems to support this result in the general case. In light of this, the authors propose an approach, JointRepair, to reduce the disparity gap introduced by coordinating predictions from multiple agents. They follow this with experiments both via traditional predictive agents (in this case, logistic regression models) and small-scale open-weight LLMs and demonstrate that JointRepair produces competitive accuracy with smaller observed and/or worst-case disparity gaps compared to other coordination methods. Lastly, they conclude with a short discussion and limitations of their work, including a note about optimization being exponential in the size of the set of coordinating agents.
Critical strengths
This paper uses both theoretical and empirical rigor to prove an interesting and important result pertaining to algorithmic fairness and demographic parity in the generative AI era. Results are thus supported with proofs and derivations in appendices, and experimental settings are similarly detailed in supplementary materials. Figure 1 and corresponding analysis in the Introduction are additional helpful in motivating the problem at hand.
Critical weaknesses
In my opinion, the primary weakness of this paper is that it is dense in that the prose is hard to read and the math is hard to parse. Bluntly, it could very well be the case that I do not have the expertise required to fully grasp the theoretical components of this work (my own contributions tend to be more empirical, and my background is not in LP optimization though I am vaguely aware of various related concepts), but regardless, Sections 3 and 4 were difficult to understand at a deep level.
Outside of this, the experiments were structured well but could potentially be improved in a few ways. First, I would have liked to see more about the synthetic data setup in the main body of the work. Moreover, contemporary research has noted issues with the Adult dataset (see Ding et al. (2021) "Retiring Adult") that the authors may want to consult for improved evaluations. Lastly, experiments with at most 5 agents and few public datasets were described in the main body of the work, and the LLM agent experiments used smaller and older agents compared to what is currently available via open-source providers (e.g., Qwen2.5-7B-Instruct compared to GLM-4.7-Flash). To the extent possible (as the exponential runtime constraint may make things difficult) running experiments with more logistic regression agents, more public datasets (e.g., datasets described here or in the folktables package described in the link above) and more and/or larger LLMs could possibly be beneficial in illustrating the theoretical result.
Questions for authors
Strengths and weaknesses noted above, I have the following remaining questions for the authors:
- Does this (dis)parity result hold for other definitions of fairness (e.g., equal opportunity)?
- Have you considered settings beyond the binary attribute setup (e.g., ones that reflect multiple races or the consideration of race and gender simultaneously)?
- I may have missed this, but are the LLM agents calibrated like the logistic regression models on producing fair outputs at the individual level (before coordination)? The crux of this paper is that fairness at the individual level can still result in unfairness after coordination, and both the main body and appendices describing LLM experiments refer to fit and evaluation examples, but particular fine-tuning and calibration details are missing, so I am wondering whether the LLMs are similarly calibrated. Elucidation on this would be appreciated.