Do third-party frontier AI evaluations matter? Analyzing publicly available evidence of company responses to evaluations

Government AI institutes and independent evaluators study frontier AI systems to identify, evaluate, and better understand potential risks. However, identifying and evaluating a risk itself does not mitigate it. Here, we examine publicly documented company and policy responses to third-party evaluation findings of frontier AI systems. We identified 1,146 distinct findings about frontier models across 457 AI evaluation reports published by government institutes, academia, and evaluation organizations between March 2023 and August 2026. Of these, we classified 233 as adverse empirical findings concerning identifiable models or developers for which a company response could be reasonably expected. Among 190 findings that met predefined significant-risk criteria, 152 findings (80%) lacked a specific and proportionate publicly documented company response for risk mitigation. Collectively, the results suggest that, based on available information, there is limited public evidence that third-party evaluations of frontier AI systems consistently lead to proportionate, attributable, and verifiable responses. Comparing the accountability structures currently in place of third-party evaluation organizations with those of established oversight bodies in other industries, including food, drugs, energy, transportation, and finance, we conclude that formal mechanisms for access, response, remediation, verification, and follow-up inspired by governance in these other sectors could increase the consistency with which AI evaluations lead to more proportionate action.