OvERALE: A Heuristic Approach to Pluralism Evaluation

Large Language Models (LLMs) have long been criticised for their unidimensionality and systemic biases. Incorporating such systems into society risks deepening ideological polarization and fostering tunnel vision. We require responses that balance diverse perspectives while carefully filtering out viewpoints that fall outside the bounds of societal acceptability. A term used to describe this notion is *Overton Pluralism*. This domain is still in its nascent stage and suffers from critical bottlenecks, such as the lack of a reasonably defined Overton window, the disruption of objective evaluation by long-form text, and the limitations of human-in-the-loop evaluation that constrain progression towards generalizability and efficiency in terms of cost and speed. In our attempt to address these limitations, we propose **OvERALE** (**Ov**erton-ground**E**d, **R**anking-**A**ligned corpus **L**attice **E**valuator), a framework that assigns a score to long-form responses, used as a proxy for pluralism. It is assessed by projecting the Overton window of individual human viewpoints (*Micro* lens) into a topic-level domain (*Meso* lens), situating pluralism within this broader topic space for downstream evaluation. OvERALE trains a rank-consistent pluralism lattice over a static gold corpus, which then serves as a static reference for evaluating new responses. Empirical sensitivity analysis shows OvERALE's score moves reliably in the expected direction across targeted perturbations, outperforming diversity and coverage based baselines as well as LLM-as-judge evaluators in directional alignment and stability. Our framework marks a meaningful step toward automated Overton pluralism evaluation, offering a scalable alternative to human annotation that extends naturally to large bodies of text.