Discrete Bridges for Mutual Information Estimation

Diffusion bridge models in both continuous and discrete state spaces have recently become powerful tools in the field of generative modeling. In this work, we leverage the discrete state space formulation of bridge matching models to address another important problem in machine learning and information theory: the estimation of the mutual information (MI) between discrete random variables. By framing MI estimation as a domain transfer problem, we construct a Discrete Bridge Mutual Information (DBMI) estimator suitable for discrete data, that poses difficulties for conventional MI estimators. We showcase the performance of our estimator on three MI estimation benchmarks: low-dimensional, DNA with low-dimensional tags and image-based.