- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses a meaningful problem: how to answer natural-language questions over heterogeneous enterprise tables when explicit schemas, reliable column names, and normalized layouts are unavailable. It proposes TACKLE, which first induces a conceptual knowledge graph (KG) from tables and then answers questions through a two-stage NL-to-Cypher pipeline. It also introduces ERP-QA, a configurable benchmark-generation framework.
Critical strengths
I would recognize the following contributions.
- A relevant problem formulation. The paper focuses on a practical gap between clean-schema benchmarks and messy enterprise data.
- A coherent system design. The conceptual KG provides a reusable intermediate representation, while decoupled query generation separates semantic decisions from syntax.
- Promising empirical results. TACKLE improves on its own direct-generation variant and several representative baselines across multiple LLM backbones.
- An explicit benchmark-generation framework. ERP-QA provides controllable table structures and deterministic answer verification, which could be useful for future research.
- Useful failure analysis. The paper identifies concept clustering, entity merging, data inconsistency, and missing background knowledge as important failure modes.
Critical weaknesses
Limited methodological novelty. The principal components—LLM-based semantic understanding, column clustering, entity resolution, relation induction, and decomposed query generation—are largely established techniques.
Insufficient real-world validation. ERP-QA is small and synthetic, while Practical-MMQA mainly simulates schema removal and name misalignment.
Baselines may be underpowered. The lightweight Text2SQL implementation and limited comparison against advanced schema-discovery and multi-hop reasoning methods weaken the empirical claims.
Incomplete ablations. The experiments do not adequately isolate the contribution of individual KG-induction components or compare against an oracle KG.
Limited scalability and cost analysis. The paper does not sufficiently quantify the upfront cost of KG construction, update overhead, latency, and performance on substantially larger enterprise environments.
Questions for authors
Q1. What is the key methodological novelty beyond combining existing techniques?
Can the authors identify a specific algorithmic insight that distinguishes TACKLE from prior work on semantic table interpretation, schema discovery, entity resolution, and ontology-based data access?