Structured Semantic Evidence for Managing Uncertainty under Sparse ASL Sensing

Perception-driven communication systems often operate under uncertain evidence because sensing can be incomplete and recognition models can make errors. A single predicted sequence or confidence score therefore cannot reveal which parts of the intended meaning are actually supported. We frame this as a context-conditioned semantic evidence problem: given Top-K recognition hypotheses, their relative probabilities, and dialogue context, infer which semantic information is required, determined, uncertain, or missing in the current conversational turn. We introduce a slot-specific cross-attention model that jointly reasons over recognition hypotheses and context to construct a structured semantic evidence representation: a collection of per-slot records indicating which meaning dimensions are required and whether their current evidence is Determined, Uncertain, Missing, or NotRequired. Experiments on ASL as a challenging testbed show that both recognition evidence and dialogue context make complementary contributions to structured semantic evidence estimation, while poorer recognition quality is consistently associated with weaker semantic recovery. Controlled context interventions further improve English meaning recovery when the correct conversational context is available. These results support structured semantic evidence as an intermediate representation between uncertain recognition and downstream conversational reasoning.