State Correction in Visual Control: Contact Mechanisms and Task Utility

Contact can disrupt the internal state used by a visual controller even when its dynamics parameters are correct. We study how this mismatch affects task performance through controlled interventions in PointMaze, combining a fixed visual world model with calibrated physical priors. Privileged velocity feedback sharply reduces action-adapter trajectory error, whereas the tested increase in endpoint command freedom has little effect. An observation-based repair improves local prediction in development tests, but a separate task evaluation finds little additional goal occupancy over gain scaling. Physical trajectory scoring produces much larger gains, with direct planning achieving slightly higher occupancy at lower cost than reranking neural proposals under the same prior and objective. An independent PointMaze study finds modest gains from visual state resets when the planning model omits contact, while an intact contact prior remains a strong reference. Component interventions distinguish the effects of position and velocity updates. An exploratory analysis of 23 complete PushT scenes shows higher mean object coverage with visual resets and different responses to velocity replacement, extending the analysis to object manipulation. Together, these studies characterize the task value of contact-state correction and show why hybrid visual controllers should be evaluated against direct planning with the same physical information.