Identifiability of Temporal Action Structure under Incomplete Observations

We study when temporal action structure is identifiable from incomplete observations that can make missing evidence resemble genuine action change. We represent this structure by temporal boundaries and a cross-time category equivalence relation, distinguishing unique recovery from boundary bands and ambiguous relations. Our geometric analysis decomposes normalized masked affine-subspace distances into retained residual energy and coefficient-refitting leakage. Explicit conditioning and coherence assumptions yield residual-order guarantees; temporal margins then characterize when neighboring evidence helps and when boundary contamination reverses its benefit. Deterministic perturbation bounds connect ideal candidates to noisy, estimated models, while indistinguishable observations expose algorithm-independent limitations. TSCC implements the resulting masked-distance objective with adaptive temporal neighborhoods and safeguarded alternating updates. Existing experiments on five motion benchmarks support temporal aggregation under a supplied category count. The guarantees are conditional: neither objective descent nor a single predicted segmentation establishes unique recovery under arbitrary missingness.