ReCCon: Mitigating LLM Hallucinations via Fine-Grained Cross-Chain Consistency

In this paper, we propose ReCCon, an LLM hallucination mitigation framework that combines retrieval-augmented generation (RAG) with consistency analysis across multiple sampled answers and their reasoning chains. Existing self-consistency methods operate at a coarse-grained level, simply aggregating sampled final answers into a single prediction without pinpointing and inspecting the specific reasoning steps at which inconsistencies arise, thereby discarding valuable information for assessing the trustworthiness of each answer. Yet, performing fine-grained consistency analysis at the reasoning-step level requires reasoning over complex logical dependencies among the reasoning steps. ReCCon addresses this challenge by leveraging the increasingly strong capabilities of LLMs for analyzing complex logical relationships. Specifically, it employs a multi-step, LLM-driven procedure in which inconsistent reasoning steps are first identified by an LLM, and their truthfulness is then assessed through a dual-layer verification strategy combining RAG and self-consistency. Each sampled answer is subsequently reweighted based on the estimated truthfulness of its sub-claims before the answers are aggregated into a single prediction. Our evaluation shows that ReCCon outperforms RAG- and consistency-based baselines in assessing the truthfulness of both factual and predictive statements, while providing better interpretability.