acceptodds
Withdrawn from ICLR 2027

TACKLE: QUESTION ANSWERING OVER PRACTICAL ENTERPRISE TABLES VIA CONCEPTUAL KNOWLEDGE GRAPHS

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Tabular data is a primary source of enterprise knowledge, yet it is typically scattered across independently maintained tables. While large language models enable natural-language question answering over tables, existing approaches—such as formal query-based and retrieval-based methods—rely heavily on explicit relational schemas, well-aligned and clear column names, and normalized layouts. These assumptions, however, often do not hold for practical enterprise tables, making enterprise reasoning extremely challenging. To bridge this gap, we propose TACKLE, a framework that operates independently of any predefined schema or ontology. TACKLE directly induces a conceptual knowledge graph from practical tables by clustering related columns into enterprise concepts and establishing explicit semantic relations. Building upon this graph-based representation, TACKLE employs a decoupled NL-to-Cypher translation pipeline that separates high-level semantic planning from low-level syntactic rendering to improve the query reliability. We further introduce ERP-QA, a controllable and extensible benchmark generation framework that reproduces these practical table characteristics and provides deterministically verified answers. Experiments on an evaluation set instantiated from ERP-QA and a practical variant of MMQA show that TACKLE consistently outperforms representative formal query-based and retrieval-based baselines across multiple LLM backbones. Ablation studies further support the benefits of the conceptual KG and decoupled translation for practical multi-table QA. Our source code is available at https://anonymous.4open.science/r/TACKLE-4798/README.md.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Enterprise Reasoning, Question Answering, LLM

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_U16mOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses a meaningful problem: how to answer natural-language questions over heterogeneous enterprise tables when explicit schemas, reliable column names, and normalized layouts are unavailable. It proposes TACKLE, which first induces a conceptual knowledge graph (KG) from tables and then answers questions through a two-stage NL-to-Cypher pipeline. It also introduces ERP-QA, a configurable benchmark-generation framework.

Critical strengths

I would recognize the following contributions.

  1. A relevant problem formulation. The paper focuses on a practical gap between clean-schema benchmarks and messy enterprise data.
  2. A coherent system design. The conceptual KG provides a reusable intermediate representation, while decoupled query generation separates semantic decisions from syntax.
  3. Promising empirical results. TACKLE improves on its own direct-generation variant and several representative baselines across multiple LLM backbones.
  4. An explicit benchmark-generation framework. ERP-QA provides controllable table structures and deterministic answer verification, which could be useful for future research.
  5. Useful failure analysis. The paper identifies concept clustering, entity merging, data inconsistency, and missing background knowledge as important failure modes.
Critical weaknesses

Limited methodological novelty. The principal components—LLM-based semantic understanding, column clustering, entity resolution, relation induction, and decomposed query generation—are largely established techniques.

Insufficient real-world validation. ERP-QA is small and synthetic, while Practical-MMQA mainly simulates schema removal and name misalignment.

Baselines may be underpowered. The lightweight Text2SQL implementation and limited comparison against advanced schema-discovery and multi-hop reasoning methods weaken the empirical claims.

Incomplete ablations. The experiments do not adequately isolate the contribution of individual KG-induction components or compare against an oracle KG.

Limited scalability and cost analysis. The paper does not sufficiently quantify the upfront cost of KG construction, update overhead, latency, and performance on substantially larger enterprise environments.

Questions for authors

Q1. What is the key methodological novelty beyond combining existing techniques?

Can the authors identify a specific algorithmic insight that distinguishes TACKLE from prior work on semantic table interpretation, schema discovery, entity resolution, and ontology-based data access?

Reviewer_VdBgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TACKLE for question answering over multi-table enterprise data without predefined relational schemas or ontologies. It constructs a conceptual knowledge graph through LLM-based table understanding, column clustering, cross-table entity merging, and relation induction, then translates questions into Cypher using separate semantic-planning and query-generation stages. The paper also introduces ERP-QA, a configurable benchmark generation framework. Experiments on ERP-QA and a modified MMQA benchmark show improvements over several query-based and retrieval-based baselines.

Critical strengths
  1. Consistent empirical improvements. TACKLE achieves the highest overall accuracy across all three LLM backbones on both Practical-MMQA variants and ERP-QA. On ERP-QA, it also performs particularly well on multi-hop questions and nested aggregation, suggesting that explicit conceptual relations can help with questions requiring information from multiple tables (Tables 1–2).
  2. Evaluation beyond answer accuracy. ERP-QA retains the generating KG and source-column mappings, allowing the authors to evaluate the recovered entities, properties, relations, and relation properties. This provides useful evidence about the KG induction stage independently of downstream QA accuracy (Table 3).
Critical weaknesses
  1. The experiments do not fully isolate the contribution of the conceptual KG. The Text2SQL baseline uses a lightweight implementation without schema linking, decomposition, or iterative refinement, while TACKLE benefits from an LLM-based semantic induction pipeline. The comparison therefore does not establish that the conceptual KG is better than a relational representation equipped with comparable inferred column descriptions and cross-table correspondences. Likewise, TACKLE versus TACKLE-naive shows that the complete decoupled pipeline performs better than direct Cypher generation, but does not establish that the gain comes from the decomposition itself rather than the additional LLM call or inference budget. Matched controls are needed to support the claims in Section 5.3.
  2. The ERP-QA aggregate scores are inconsistent with the reported category scores. Using the question counts in Table 5 to weight the five category accuracies in Table 2 gives approximately 72.08%, 79.69%, and 82.50% for GPT-5.4-mini, DeepSeek-v4-flash, and Gemini-3.6-flash, respectively. The reported totals are 72.8%, 80.4%, and 83.0%. The differences are too large to be explained by one-decimal rounding alone. The authors should explain the aggregation rule and provide per-run counts and denominators.
  3. The evaluation does not yet establish broad generalization to practical enterprise data. ERP-QA evaluates 334 questions from only two instantiated scenarios, and its tables and questions are generated from predefined schemas and templates. Practical-MMQA retains the original data and questions while changing table and column names and removing explicit schema information. These evaluations are useful controlled tests, but do not fully demonstrate robustness to real enterprise data inconsistencies, ambiguous values, or previously unseen domains.
Questions for authors
  1. How are the ERP-QA Total scores calculated? Please reconcile the reported totals with the category scores and counts in Tables 2 and 5. Provide per-run correct-answer counts, denominators, and any excluded questions.
  2. Does TACKLE still outperform a relational baseline that receives comparable LLM-generated column descriptions and cross-table correspondences? Can the decoupled translator outperform direct generation under a matched call or token budget?
  3. For Practical-MMQA, are the same question IDs used across methods and backbones? Are the three runs based on different subsets, and is KG construction repeated for each run? Please report the sampling protocol, decoding settings, and paired uncertainty for TACKLE versus TACKLE-naive.
Reviewer_GT7COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper formulated a multi-table QA over practical enterprise tables characterized by being schema-free, semantically unaligned, cryptic and denormalized and proposed a two stage framework that induces and leverage the conceptual structure of practical enterprise tables. To evaluate the effectiveness of the method, the author also proposed a ERP-QA benchmark generation framework. Experiments on two datasets show that TACKLE consistently outperforms the baselines across different LLM backbones.

Critical strengths
  1. The paper studied an interesting and practical problem. Instead of focusing on the normal multi-table QA, the author focused on the tables characterized by being schema-free, semantically unaligned, cryptic and denormalized.
  2. The method design is straightforward and tightly coupled with the problem itself.
  3. The proposed method consistently outperforms the baselines, especially on more complex reasoning tasks.
Critical weaknesses
  1. The effectiveness of the conceptual KG and what-how decomposition is underexamined and unclear. In table 2, the author compared the Text2SQL and Tackle-naïve to show the effectiveness of KG, but the difference between these two parts is more than just the KG. Performance improvements may stem from a combination of LLM semantic annotation, pre-established join relationships, and differences in query languages (SQL vs. Cypher); Similarly, it is unclear whether the improvement from the what–how decomposition comes from the proposed design or simply from using an additional LLM call.
  2. Since the overall pipeline consists of multiple LLM calls. The cost and scalability analysis is missing which is important to assess the performance and cost trade-off using this method compared with other baselines.
  3. I am confused about the practicality of the method since (1) The benchmark the author used is generated from a conceptual KG, then evaluated on a KG based method. How many of the practical tables following the underlying KG construction assumption/relationship? (2) How representative are the two benchmarks of real-world enterprise tables, did you test or study the cases when there are ambiguous identifiers, missing values etc.?
Questions for authors

Please see weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.