MorphState: Coupling State Aggregation and Detail Recovery for Fine-Grained Segmentation

State space models represented by Mamba achieve efficient long-range modeling through selective state updates. However, for visual models that aggregate spatial features into a small number of states, deciding which information should be written into the states does not directly reveal which local structures remain insufficiently represented, leaving subsequent detail recovery without an explicit reference to the current aggregation. To address this issue, we propose MorphState, a fine-grained segmentation framework that uses the spatial allocation of state writes as a shared reference for global aggregation and for generating recovery candidates through regularized feature reconstruction. First, high-resolution morphological cues modulate the spatial write weights while keeping the state count fixed. Next, the updated allocation is treated as a spatial reconstruction dictionary to regularize the reconstruction of pre-aggregation features, and the components that cannot be sufficiently explained by this dictionary are extracted as complementary candidates. Finally, because such residual components may also contain irrelevant texture variations, the model combines morphological orientation, residual magnitude, and coarse semantic predictions to modulate their class-specific contributions and refine high-resolution segmentation outputs. On BD3-Seg, MDMCS, and CUBIT-Seg, MorphState achieves a five-class mIoU of 85.77%, and foreground mIoUs of 83.45% and 53.07%, respectively, outperforming the strongest compared methods on the three datasets by 5.00, 4.32, and 4.18 percentage points. Ablations show additional gains from the complete allocation-and-recovery configuration over MGCR alone on BD3-Seg and MDMCS.