F2: Offline Reinforcement Learning for Hamiltonian Simulation via Free-Fermionic Subroutine Compilation

Compiling shallow and accurate quantum circuits for Hamiltonian simulation is challenging due to hardware constraints and the combinatorial complexity of minimizing gate count and circuit depth. Existing compilers rely on hand-engineered heuristics and cannot adapt to input-dependent structure, limiting optimization opportunities. We introduce F2, an offline reinforcement learning framework that exploits free-fermionic structure for quantum simulation. Across benchmarks spanning lattice models, protein fragments, and crystalline materials (12–222 qubits), F2—using a single trained model—reduces $R_Z$ count by 28.8% on average relative to strong industry and academic baselines (IBM Qiskit, Google Cirq/OpenFermion, Berkeley BQSKit), while maintaining errors on the order of $10^{-7}$. These results demonstrate that aligning reinforcement learning with algebraic structure can substantially improve scalable quantum circuit synthesis.