Seam-DETR: Transition- and Quality-Aware Decoder for Temporal Action Detection

Query-based detectors have become a strong family for temporal action detection, yet their localization quality remains unsatisfactory. We trace the problem to the two sides of the decoder. On the input side, the memory attended by the decoder encodes temporal boundaries only implicitly, making it difficult to perceive temporal transitions, especially gradual ones. On the supervision side, one-to-one matching assigns the positive label to a single proposal per instance and treats its near-duplicates as background, so the learned scores only reflect the matching assignments rather than the actual overlap with the ground truth. We propose Seam-DETR, which restores localization quality from both sides. Transition-Aware Memory (TAM) introduces two parameter-free views of the memory to what the localization attention reads: one measuring how much the content changes at each moment and the other exposing what it changes into, so the transitions become perceptible. Match-Aware Ranking (MAR) prevents unmatched near-duplicates from being suppressed as background and ranks same-class proposals by their overlap with the ground truth, making confidence scores more localization-aware. The two designs improve localization quality with minimal overhead, where TAM adds only two lightweight adapters, and MAR introduces no additional parameters or inference cost. Extensive experiments demonstrate that Seam-DETR achieves state-of-the-art performance among query-based methods on THUMOS14, ActivityNet-1.3, and HACS-Segment.