AMBER: Anytime Multimodal Memory Bridging Episodes via Rich Association

Multimodal conversational memory systems struggle with required evidence when information is updated, extended, or contradicted across episodes, where previous similarity-based retrieval failed. To address this challenge, we introduce AMBER (Anytime Multimodal Memory Bridging Episodes via Rich Association), which builds graph with temporal, procedural, complementary, and conflicting associations, continually organizing episodes into paths centered on shared association anchors. When association retrieval is needed, AMBER adaptively activates one type for the query, traverses matching paths from initial semantic matches, and ranks the resulting evidence chains under a fixed budget. To evaluate whether memory systems can preserve required evidence while incorporating new information, we introduce MemStream, annotated 1,603 information states for 779 evolution-related questions with reference answers and supporting evidence from Mem-Gallery and H2HMem. MemStream enables streaming evaluation with process-level metrics for answer continuity and degradation, revealing an average 58.48% decrement in state F1 compared to the final. Across 2 widely used MM benchmarks and our proposed one, AMBER improves average F1 by 2.68% over the strongest of four baselines, while recovering 19.07% more evidence in cross-episode queries.