Wallbreaker: Breaking the Vision-Imagery Barrier through Cortical Hierarchy-Driven Neural Decoding

Decoding mental imagery from fMRI is fundamentally challenging: models trained on visual perception degrade sharply when applied to imagery. We argue that this gap is not merely a signal-to-noise problem, but stems from a hierarchy-structured representation shift in the visual cortex. Vision is primarily driven by bottom-up sensory input, whereas imagery is shaped by top-down internal generation, systematically redistributing information across cortical levels. This induces two forms of hierarchy-dependent variability: a distribution-level shift in the balance of cortical information between vision and imagery, and a stimulus-level preference where different stimuli rely to different degrees on structural versus semantic information. We propose Wallbreaker, a framework that explicitly models cortical hierarchy-dependent variability during neural decoding. We first introduce hierarchy-driven representation perturbation, which simulates realistic hierarchy shifts during training by redistributing and perturbing representations across cortical levels. We further develop hierarchy-conditioned reconstruction, which infers a stimulus-wise hierarchy preference to dynamically balance semantic guidance and structural anchoring during reconstruction. Wallbreaker improves key imagery reconstruction metrics while maintaining competitive vision reconstruction performance, effectively narrowing the vision-to-imagery gap. More broadly, our results highlight cortical hierarchy as a fundamental inductive bias for bridging vision and imagery in neural decoding.