Fair Ranking with Attention Uncertainty

Exposure fairness asks that each group of items in a ranked list receive a target share of user attention. Whether a ranking meets this target depends on an attention curve, which specifies how attention is distributed across positions, yet the curve used for evaluation may differ from the one users actually follow. This raises the question of what a fairness score computed under one curve, or a few, guarantees when the true attention curve is unknown. We first characterize the blind spot of a finite set of checked curves, which is the largest exposure disparity that can remain hidden while all checks pass at a given tolerance. For arbitrary targets and multiple, possibly overlapping groups, an exact formula is derived to compute the blind spot from the checked curves within the tolerance. We show that commonly used attention curves can still leave substantial exposure disparities undetected. For exact checks, we also give upper and lower bounds on the number of curves needed to guarantee a desired blind spot. Finally, we design a robust ranking linear program that maximizes utility while controlling exposure disparity under an unknown attention curve. Across Amazon ESCI, TREC Fair Ranking 2022, and German Credit, we show that policies that are fair under a single logarithmic attention curve are still off target on average by 36%–44% of total attention under some monotone attention curve. Under the worst such curve for each query, these policies deliver on average only 42%–64% of the target exposure. They remain off target by 7%–11% of total attention under attention curves estimated from real-world click logs, under which they deliver on average only 80%–91% of the target exposure. By contrast, our robust linear program keeps the mean deviation below 0.06% of total attention, so that it delivers at least 99.8% of the target exposure, with an additional loss of about 1% or less in ranking utility relative to the single-curve policies.