PHANERO: Partial-occlusion Hidden-state Alignment with Neural-dimension Evidence for Recovering Objects
Abstract
A system that already runs a vision-language model (VLM) for captioning or planning can ask it for a bounding box instead of serving a separate detector. Occlusion is what breaks that arrangement: boxes emitted as text become unreliable exactly when the object is partly hidden, and the standard repairs — prompting, supplied geometry, fine-tuning — all act before deployment. PHANERO asks what can be repaired during the decode itself. Occlusion displaces the decoder's final hidden state, and part of that displacement is linearly recoverable. From clean and occluded decodes of the image we fit one map back to the clean state and apply it through a forward hook where coordinates become tokens, keeping the unrepaired boxes alongside. Nothing is trained and no gradient is taken; the map is fit once per model on COCO under synthetic occlusion, then used unchanged on KITTI and HOOT under natural occlusion, where it raises recall at IoU () in all nine modeldomain cells and outscores published pixel-, attention- and visual-token-space repairs in nine of eighteen paired tests, losing one. Keeping the unrepaired boxes means any second decode collects part of that gain, so we pre-registered the test that separates them: the same union built from a norm-matched random write and from a resampled decode. The learned correction clears the stronger null on all three models ( to ; of cells, missing KITTI on two models and Qwen3-VL HOOT), on the weakest model partly as extra boxes. Rescaled to its edit size, maps without the pairing or the cross-dimensional terms fall short of it in of localization tests. What this buys is narrow and cheap. A stock open-vocabulary detector with a tuned threshold still beats every cell we repair; the claim is only that a VLM already in service can localize better under occlusion for one extra decode — a plain query ( by total time) with both decodes batched.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.