Capacity Is Not Accessibility in Autoencoders

The high expressivity of trained decoders in autoencoder models is usually attributed to their learned weights. However, we find that even random convolutional decoders can achieve high-quality reconstructions when sufficiently high-dimensional intermediate representations are optimized directly. This raises the question: what does learning contribute if substantial capacity is already present in random decoders? We decompose autoencoder reconstruction errors into three components: a decoder-limited error, an encoder gap between the best bottleneck state and that predicted by the encoder, and an accessibility gap between the best intermediate representation and the best state reachable through the bottleneck. Across image and audio datasets, we show that increasing bottleneck dimensionality systematically reduces the accessibility gap while increasing the encoder gap, leaving the total gap comparatively stable. Training therefore does not merely provide capacity, it organizes high-dimensional capacity so that useful outputs can be reached through far fewer control variables. We further find that the accessibility gap spans many functionally relevant decoder directions and can extend under distribution shift. Together, these results suggest that poor reconstruction through a trained decoder can reflect restricted access to its capability rather than insufficient decoder capacity.