FIRM: Morphology-Guided Residual Experts for Fine-Grained Pathology Image Synthesis

Fine-grained pathology image synthesis requires modeling both diagnostic distinctions across categories and morphological diversity within each category. However, conventional category conditioning specifies the target diagnosis while leaving within-class variation largely implicit. We introduce Feature-Informed Residual Modulation (FIRM), a framework that structures pathology representations into inter-class diagnostic variation and intra-class morphology for conditional generator adaptation. FIRM first learns a shared pathology prior and constructs a diagnostic and morphological geometry from fixed training-image features, decomposing within-class morphology into discrete modes and continuous residuals. These structured conditions modulate class-routed low-rank residual experts, allowing each diagnostic category to specialize the shared generator while preserving fine-grained morphological variation. We further introduce paired pathology feature refinement, which aligns predicted clean images with their corresponding real training crops in feature space without additional sampling trajectories. Experiments on BRACS and SICAPv2 show consistent improvements in class-conditional distribution matching and feature-space coverage over existing generative baselines. Controlled interventions further demonstrate discrete and continuous morphology manipulation, while synthetic augmentation improves downstream classification.