Geometric Consistency of Concrete Scores in Discrete Diffusion Models

In score-based formulations of discrete diffusion models, the reverse process is parameterized by probability ratios between neighboring states, commonly referred to as the concrete score. We characterize this score through discrete differential geometry on the graph induced by the transition structure. The logarithmic concrete score of any positive probability distribution forms an exact $1$-cochain: the discrete differential of log probability on vertices. Therefore, it must be antisymmetric under edge reversal and have zero circulation along every cycle. Existing discrete diffusion models do not explicitly enforce these conditions, so their learned ratios need not correspond to any single distribution. To investigate this gap, we use the weighted Hodge decomposition to separate log concrete score error into three mutually orthogonal components: antisymmetry violations, cycle inconsistency, and data-score mismatch. We investigate their contributions to squared field error and how interventions on them relate to generation. Closed-form geometric projections $P$ and a trained data-informed corrector $C$ provide complementary interventions on the same frozen backbone. Projection targets geometric inconsistency; the corrector uses data-space feedback and can change all three components. We study each operator and their composition, including projection amortization. Experiments on uniform-noise language models examine geometric residuals, data-space likelihood, diversity, and inference cost. This investigation connects component-wise geometry and generation while distinguishing field realizability from alignment with the target distribution.