TC-NMD: Trajectory-Conditioned Non-Markovian Diffusion for DNA Sequence Design

Discrete diffusion models have emerged as a competitive framework for biolog- ical sequence design, achieving strong results on DNA enhancer and promoter generation benchmarks. Yet the models driving these results, masked diffusion, Dirichlet flow matching, and the simplex-based shortlisting model (SLM), all share a common structural assumption: the reverse process is Markov, so each denoising step conditions only on the current noisy state. This is convenient, it yields tractable per-step losses and ancestral sampling, but it means the reverse step ignores the rest of the sampling trajectory, even though those noisier states are already computed during generation. Recent non-Markovian discrete diffusion models show that conditioning the reverse process on this trajectory improves generation quality. We propose TC-SLM, a non-Markovian simplex diffusion model that brings this idea to DNA sequence design. TC-SLM represents corrupted DNA as simplex-valued distributions over nucleotides and, at each reverse step, augments the denoiser with a learned summary of the cached noisier states: a Gumbel-Softmax router scores those states by their compatibility with the current state and forms a routed history context used to predict the clean sequence. On conditional enhancer and promoter design, and on unconditional protein genera- tion, a non-Markovian model trained from scratch improves over its Markovian counterpart, most substantially on fly-brain enhancer design, while adding only a lightweight router to the architecture.