A power saving for intersective polynomial differences with an exponent depending only on the degree

An integer polynomial is intersective if it has a root modulo every positive integer. For each degree k ≥ 2, we prove that there is an exponent $c_k>0$ such that every set $A\subseteq \{1,\dots ,N\}$ whose differences avoid all nonzero values $h(1),h(2),\dots$, where h is an intersective polynomial of degree k with positive leading coefficient, satisfies $|A|=O_h(N^{1-c_k})$. The implied constant may depend on h, but the power-saving exponent depends only on its degree.