Not Whether, But Which: Localizing Concept-level Knowledge Gaps behind LLM Inconsistency

Large language models (LLMs) exhibit inconsistency: semantically equivalent inputs elicit contradictory factual outputs, eroding their reliability in downstream applications. Prior work has documented this phenomenon but leaves a more fundamental question unanswered: which knowledge deficiencies produce inconsistency? Knowing that a model is inconsistent is of limited use unless we know which knowledge it lacks. In this paper, we posit that inconsistency stems from identifiable, localizable gaps in concept-level knowledge. To test this hypothesis, we introduce KnowledgeMap, a pipeline that pinpoints such gaps by automatically parsing inputs into lexical concepts, applying calibrated perturbations, and then dynamically profiling concept-level proficiency via consistency measurements. Evaluated across large-scale datasets and diverse LLMs, KnowledgeMap reliably isolates specific knowledge deficiencies and yields actionable guidance for targeted model training.