Optimal Control of a Continuum of Agents in MDPs: The Power of Finitely Many Contracts

We study how a principal can control the state distribution of a continuum of agents through dynamic contracts, implementing a prescribed trajectory at minimum expected payment. In a finite-horizon Markov model with $n$ states, agents of finitely many observable types privately choose actions from compact sets, with known, future payoff. The principal commits to randomized offers of bounded, nonnegative payments contingent on the next state. Agents are repooled by type and current state after each transition, and ties between best responses are resolved in the principal's favor. We show that finitely many contracts suffice to attain the minimum expected payment for every implementable trajectory, yielding an exact finite-dimensional formulation of the infinite-dimensional design problem. Combining backward incentive conditions with forward population flows, we obtain an optimal implementation using at most $n+1$ payment-action pairs per type, current state, and decision date. Joint reoptimization across current states reduces total support per type to at most $2n-1$ pairs at the first date and $3n-2$ at every later date, while retaining the local bound and minimum cost. This improves the support bound across current states from quadratic to linear in $n$ by preserving aggregate population flows and relative continuation values. Both per-date joint support bounds are tight, already with two decision dates and continuous scalar actions.