Unpacking Evolving Dynamics Through The Representation Pathway

Neural networks transform representations along computational pathways, via stacked layers or iterative processes such as diffusion sampling. How these representations evolve and how their evolution relates to model behavior, provides a complementary perspective to analyzing final representations alone. To this end, we introduce Representation Pathway Consistency (REPAC), an image-level analytical metric that quantifies how well relational structure among an image's patch tokens is preserved between successive pathway states and aggregates these local consistencies across the pathway. Using REPAC, we find that higher pathway consistency is generally associated with better performance across discriminative and generative tasks. In generative models, generated images with more consistent pathways tend to lie closer to class-conditional prototypes in representation space, providing a geometric perspective on this association. Moreover, REPAC computed from an initial segment of a generative pathway is predictive of final sample quality. Leveraging this signal, we develop a plug-and-play early-selection strategy that ranks partially generated candidates using REPAC and continues only the highest-ranked ones to completion. This reduces sampling cost while improving the quality of retained samples, without external reward signals or additional training. Together, these findings reveal regularities in representation pathway dynamics and their relationship to downstream behavior, while demonstrating the utility of pathway consistency for quality-aware and efficient selective generation.