- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes ComInsight for table-to-report generation. It enumerates candidate atomic findings using analytical patterns and SQL templates, filters them using statistical tests and a learned interestingness judge, and organizes them into a multi-relational graph. Composition procedures generate higher-order findings, which are synthesized into reports with SQL provenance. Evaluation covers InsightBench, DDR-Bench, and T2R-Bench.
Critical strengths
-
The motivation goes beyond generating fluent reports: an analysis should discover related findings across tables and organize them into traceable evidence chains. The example of sequential exploration becoming anchored on early local findings makes structured composition a relevant direction.
-
Explicit SQL provenance and the separation of atomic evidence from report synthesis are promising design choices. The reported gains across three benchmarks and the included ablations suggest that the integrated pipeline has practical potential.
Critical weaknesses
-
The distinctive contribution of structured composition is not sufficiently established. Atomic finding enumeration, insight filtering, relation graphs, and LLM synthesis each have precedents. Their integration could still be a contribution, but the paper needs to demonstrate what new analytical capability the composition framework enables and why it remedies exploration bias. The current comparison with simple LLM concatenation does not adequately isolate structured composition from the availability of more candidate findings, intermediate organization, or additional computation. A graph-free composer supplied with the same verified atoms and a comparable budget would be a more informative test. This is more important than claiming that the individual components must themselves be new.
-
The evidence does not yet support the central claim of fully verifiable composite conclusions. Executing the SQL for constituent atoms establishes those observations, but not necessarily the relationships or explanatory statements synthesized from them. The main text appropriately calls causal composites hypotheses, whereas the abstract and appendix use stronger language about full verifiability and causal verification. Granger tests and an LLM plausibility judgment do not alone establish a causal explanation. The evaluation should distinguish reproduction of atomic values, correctness of composed relational claims, and support for causal interpretations. The mentioned manual auditing does not provide enough quantitative evidence to resolve this gap.
-
The learned filtering stage introduces an important evaluation uncertainty. Training examples for the interestingness judge and calibration are drawn from the same named benchmarks used in evaluation. This does not establish leakage by itself, but the exact separation of training, calibration, and test tables/tasks needs to be explicit. An independent evaluation of the judge is also needed to support the claim that filtering approximately 85% of candidate atoms preserves valuable discoveries. Otherwise, the completeness and accuracy gains cannot be interpreted confidently.
Questions for authors
- What analytical capability is enabled specifically by the proposed graph and composition operators, compared with a graph-free composer given the same atoms, provenance, and budget?
- Can the authors report a claim-level audit separating correct atomic evidence from correctly supported composite relationships? How are causal hypotheses distinguished from verified factual conclusions in the output and evaluation?
- What exact table/task-level split separates judge training and threshold calibration from benchmark testing, and how well does the filtering stage retain important findings on held-out data?