Knowledge, Mechanisms, and Context: Diagnosing and Improving Social UGC Understanding — @SITU

Understanding social user-generated content (UGC) requires interpreting evolving expressions and integrating meaning across posts and interactions. Existing benchmarks largely evaluate downstream task labels or individual phenomena, leaving these abilities difficult to assess together. We introduce SITU (*Situated Interpretation of Textual UGC*), a bilingual benchmark built from Chinese and English social-media posts and discussions. It contains 89,199 open-ended interpretation probes from three platforms, including 9,522 difficulty-filtered probes on which frontier models remain well below perfect performance. Nine coverage-oriented probe targets span expression-centered and context-integrative challenges. To study improvement, we analyze item knowledge, reusable expression mechanisms, and contextual integration through retrieval, expression oracles, supervised fine-tuning (SFT), and training-data scaling. Retrieval helps most when it supplies answer-critical knowledge; SFT produces broader gains, and their combination performs best overall. A full-stream Chinese exposure audit finds persistent gains on mechanism-transferable expressions after excluding detected same-item knowledge. Matched downstream training further shows benefits for translation and homophone-robust toxicity detection, with substantial variation across tasks and backbones. These results establish SITU as a testbed for evaluating social UGC understanding and identifying which improvements can transfer beyond interpretation. We will release the benchmark and evaluation code.