Generative Replay Mitigates Sample Starvation in Quantum Architecture Search

Reinforcement learning (RL) can automate quantum architecture search (QAS), but its scalability breaks down when useful circuit trajectories become rare in a rapidly expanding search space. We show that the probability of encountering an $ϵ$-accurate low energy circuit through passive exploration decreases super exponentially with qubit count, making uniform and prioritized experience replay fundamentally insufficient for large-scale QAS. We introduce GenQAS, a tensor network-guided RL framework that combines a fixed matrix product state warm start with prioritized generative replay. In this framework, a learned local transition model generates synthetic circuit transitions on demand from real state-action interactions and mixes them with real experience during Double Deep Q-Network updates. Across chemical Hamiltonian benchmarks from 6 to 12 qubits, GenQAS improves fixed-budget success probability and identifies more compact circuits at competitive energy error, improving final success probability by up to 7.0$\times$ over passive replay at 12 qubits. On a 15-qubit transverse field Ising model, GenQAS reaches 21% success probability, a 75% relative gain over uniform TensorRL-QAS (12%) and a 40% relative gain over prioritized TensorRL-QAS (15%). To test transfer, we move a replay buffer trained in a noiseless 6-qubit ${\text{BeH}}_2$ environment into a noisy version of the same task, comparing a transferred uniform buffer against a transferred GenQAS generative buffer. The uniform transfer reduces steps to chemical accuracy by only 30.1%, whereas the GenQAS generative replay transfer reduces steps by 92.7%, a substantially larger improvement under identical noise conditions. These results show that generative replay mitigates sample starvation in QAS and, critically, that the learned generative buffer carries far more transferable structure than passive replay, supporting more resource-efficient and robust circuit discovery. The code is available at https://anonymous.4open.science/r/GenQAS/README.md.