Decoding under Constraint Uncertainty in Diffusion Language Models
Abstract
Constrained decoding usually assumes a known grammar, schema, or rule set. When constraints must be inferred from positive demonstrations, several hypotheses can explain the same observations, and enforcing one can exclude the correct output. We formalize this setting as hidden-constraint decoding and introduce TACTIC, an execution layer that carries constraint uncertainty into candidate selection. TACTIC converts posterior beliefs or confidence-weighted rules into candidate compatibility and combines it with frozen-model evidence. Factorized and tensor-train representations support independent and correlated constraint components. Our analysis establishes a false-exclusion lower bound for point commitment, posterior support preservation, and conditions for decision stability. Across controlled and real-data tasks, TACTIC improves on matched hard execution under scarce support. On CIDDS structured completion, the gains are 19.4 percentage points with DREAM-7B and 20.8 with LLaDA-8B. These results show how retaining plausible constraints allows model evidence to resolve ambiguity left by demonstrations.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.