Posterior-Mean Rectified Flow to estimate 3D structures from 2D medical images

This study addresses the highly ill-posed challenge of 3D volume reconstruction from 2D medical images even in scenarios lacking a known physical forward model. We propose a two-stage framework that first employs a specialized “lifting” network to generate a structurally consistent Minimum Mean Squared Error (MMSE) estimate. This smooth baseline then serves as the foundation for a 2D paired flow-matching process, designed to recover high-frequency details across large 3D volumes efficiently. By decoupling structural translation from perceptual refinement, we leverage the generative power of flow matching in 2D while retaining structural consistency through spherical linear interpolation (slerp) without the need for explicit data-fidelity terms. Our approach operates directly in image space, bypassing the ambiguity of latent space correspondences and enabling test-time control over the distortion-perception tradeoff via adjustable refinement steps. When evaluated on the translation of synthetic X-ray to Computed Tomography (CT) and real Color Fundus Photography (CFP) to Optical Coherence Tomography (OCT), our method demonstrates superior structural fidelity and a significantly lower FID compared to state-of-the-art baselines. Code and example GIFs are available here: https://anonymous.4open.science/r/pmrf3dsynthesis-CEC5.