KnoGML: Knowledge-Grounded Multimodal Learning for Fault Diagnosis

Reliable fault diagnosis is essential to industrial safety and often relies on fragmented, variable-length time series. Many existing methods achieve diagnosis by learning a mapping from numerical features to fault labels, but face two key limitations: 1) they typically rely on lossy operations such as truncation or downsampling to accommodate fixed-length model inputs, limiting the use of complete recorded segments; 2) the domain-specific physical meaning of signals and their relationships is not explicitly utilized. To this end, we propose KnoGML, a knowledge-grounded multimodal learning framework that combines multivariate plot images with structured diagnostic priors for fault diagnosis. Specifically, domain-specific fault knowledge is organized into structured prompts describing variable meanings, cross-variable relationships, and fault associations within a three-stage diagnostic protocol comprising phase identification, symptom verification, and final decision. Meanwhile, multivariate time-series data are converted into fixed-resolution grid images, providing a unified visual representation for time-series segments of varying lengths. Building on this, a multimodal large language model jointly processes the structured prompts and visual representation, transforming fault diagnosis into a matching problem between fault descriptions and signal patterns. Finally, a classification head produces a deterministic fault label, providing a fixed output interface for industrial integration. In electric vehicle battery fault diagnosis, KnoGML improves Macro-F1 over the best traditional baseline by 7.44 and 1.97 percentage points across two battery chemistries, demonstrating the effectiveness of incorporating structured fault knowledge into multimodal learning. Code is available at https://anonymous.4open.science/r/KnoGML-77AB/.