When Semantics Stay the Same but Predictions Change: Auditing Checkpoint Portability under Local Nominal Re-serialization

Aggregate task scores can conceal substantial changes in a model’s individual de- cisions. We study this reliability gap for conversation-local nominal identifiers, where two preprocessing pipelines may assign different integer names to the same local participant partition without changing the underlying task semantics. Nev- ertheless, a frozen checkpoint using absolute-ID embeddings can produce differ- ent predictions across such semantically equivalent re-serializations. We formalize the required invariance as semantics-preserving checkpoint portability and intro- duce a semantics-gated audit that first verifies preservation of the task-semantic object and then compares paired executions of the same frozen checkpoint at the decision, confidence, and unit-risk levels. The audit distinguishes three in- terface regimes: absolute-ID lookup, empirical mitigation through augmentation or JS consistency, and exact-contract canonical or relation-only representations. On held-out ECF Pair/Query evaluation, absolute-ID lookup produces unit-level portability risks of 0.6407 and 0.6103 while ∆F1 is only −0.0038 and +0.0001, showing that nearly unchanged aggregate performance can coexist with substan- tial prediction-level divergence. Derived-ID MRDA and Entity Relation Classi- fication reproduce the same qualitative pattern across task and identifier settings. Empirical mitigation improves selected portability metrics but does not yield exact portability, whereas canonical and relation-only interfaces achieve exact consis- tency, zero confidence drift, and zero portability risk under their implementation contracts. These results demonstrate that aggregate task performance alone does not certify frozen-checkpoint portability and motivate semantics-aware, interface- level reliability evaluation.