Semantic Fields Are Interfaces: Consumer-Typed Functional Migration for Representation Autoencoders
Abstract
Representation autoencoders (RAEs) use pretrained visual features as generative states. These dense fields also support retrieval and downstream heads, forming shared interfaces whose consumers may persist across producer releases. Producer upgrades may preserve tensor shape while changing coordinate semantics, rendering retained consumers incompatible. We study post-release field migration, in which an already-trained successor serves retained consumers through one release-wide map while the generator, representation models, and consumers remain frozen. Consumer-Typed Functional Migration (CTFM) learns corrections around a host-specific calibrated base and supervises repair with archived consumer responses rather than field matching alone. Its contract separates behavior to recover from coordinates to preserve: IQ-Safe repairs decoding while exactly preserving the base-calibrated pooled coordinate, whereas Quality permits full-field correction. RAEv2 optionally employs an ownership-preserving, affine-initialized low-rank refinement. Across DINO-based RAEv2 and Perception Encoder (PE) RAEs, CTFM improves recognition, fidelity, semantic consistency, and distribution quality over matched post-hoc alternatives. On 3,232 new RAEv2 fields, a 1.05M-parameter map outperforms identical-capacity feature matching on all five primary metrics. With one quarter of the refinement samples, CTFM gains 3.87 ResNet-50 points and 1.82 dB PSNR over full-data matching; with half the PE residual-fitting classes, it gains 10.88 ResNet-50 points over the feature-to-task procedure. On 7,200 real-image queries, IQ-Safe preserves every calibrated-base retrieval decision while improving decoded PSNR by 5.95 dB. These results support consumer-defined compatibility for practical reuse of dense RAE fields under fixed consumer contracts.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.