acceptodds
Under review as a conference paper at ICLR 2027

TabClean: Evidence-Guided Program Synthesis for Scalable Tabular Data Cleaning

Abstract

Reliable analytics and machine-learning pipelines depend on clean tabular data, yet production tables contain heterogeneous errors that require different repair mechanisms. Constraint-based systems require explicit rules, while learning-based systems require labels or training. LLMs can help infer repairs, but repeatedly reasoning over cells or chunks incurs inference costs as tables grow and new batches arrive. Synthesizing executable cleaning logic offers a way to amortize this reasoning, provided that the evidence supports both the transformation and the conditions under which it should apply. We present TabClean, a model-training-free system for scalable tabular data cleaning through evidence-guided program synthesis. TabClean uses table evidence and execution feedback to synthesize guarded cleaning programs, amortizing synthesis-time LLM cost through deterministic execution over large tables and compatible batches. Table profiles and annotated examples guide the choice of repair mechanisms, target representations, and guard predicates. Cell-level feedback from executed candidates then guides program revision, and a deterministic controller retains the best development-set program. Across six benchmarks, our approach achieves a mean F1 of 0.842, improving on two LLM baselines by 0.456–0.764 with identical development examples and test rows. On a 200,000-row table, it achieves 0.983 F1 in 4.42 minutes, an approximately 1,200× speedup over a semi-supervised repair baseline. Its mean LLM API cost across the six benchmarks is 98% lower than that of an iterative LLM repair baseline.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Program Synthesis, Large Language Models, Tabular Data Cleaning。

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ZS5sOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

TabClean cleans tabular data without training a model. A profiler summarizes the table, and five role-specific agents write a Python program made of guarded clauses, each pairing a deterministic transformation with a guard that limits where it applies. Candidate programs run on annotated development rows, cell-level false positives, false negatives and regressions drive the next revision, and a rule-based decider keeps the program with the best development F1. The final program runs over the full table without further LLM calls. On six benchmarks TabClean reports a mean F1 of 0.842 and the best F1 on five, against 0.078 for Cocoon and 0.386 for AutoDCWorkflow with the same development examples.

Critical strengths
  1. LLM inference cost is amortized through program reuse. LLM calls sit in a bounded synthesis and debugging phase, and what remains is a program that can be rerun, about 0.48 dollars per dataset, 4.42 minutes on the 200,000-row Tax table, and 18 LLM calls on AddressTable against 5,915 for IterClean.
  2. Splitting a repair into a transformation and an explicit guard makes over-correction something the program can check clause by clause. The error analysis shows what unguarded edits cost Cocoon and AutoDCWorkflow under exact matching, through changes of case and date format.
Critical weaknesses
  1. The clause structure and evidence requirements are specified through prompts, without a formal mechanism enforcing structural compliance or guard correctness. The Coding Agent writes Python under a contract stated in its prompt, the guards and evidence levels are natural-language instructions in the skills and prompts, and the paper itself says this discipline does not formally guarantee correct guards or repairs. Apart from the LLM Review Agent, no component checks that generated code contains the guards it claims. The rest of the system, from the role split to the debugging loop and the execution-based selection, follows the agent literature the paper cites. What is new is a set of pattern-level rules for unseen values and cross-column clauses plus a loop that refines guards, and the paper never walks through a full synthesis episode, so a reader cannot tell this apart from careful prompt design.
  2. The standard comparison gives the systems very different amounts of supervision, so its F1 gaps do not isolate repair ability. Half of the 30 baseline entries in Table 3 round to 0.00, Cocoon scores 0.15 or lower everywhere while its own paper reports higher scores on five of the same benchmarks under a lenient protocol, and HoloClean scores 0.00 on five datasets even with oracle error locations. TabClean uses 110 to 369 annotated development rows, against 20 tuples for Baran and 5 for IterClean. Table 4 partially addresses this concern by giving Cocoon and AutoDCWorkflow the same development examples and test rows. However, comparable supervision-matched results are still missing for Baran and IterClean.
Questions for authors
  1. Why do half of the baseline entries round to 0.00, including Cocoon on benchmarks where its own paper reports higher scores and HoloClean with oracle error locations? How do the comparisons change under a matching rule that tolerates case and null-token differences?
  2. How do Baran and IterClean perform when given the same development rows as TabClean?
  3. What do the guards contribute on their own? For example, how do precision and F1 change when the guard requirements and evidence levels are removed from the prompts, and how often does each guard family appear in the generated programs?
  4. Is there evidence that a committed program stays accurate on a later batch, or on a batch drawn differently, without being synthesized again?
1 comment

The authors have not yet responded to my comments, so I have decided to maintain my original score.

Reviewer_hniaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces TABCLEAN, a multi-agent LLM framework that cleans tabular datasets by using statistical table profiles and a small set of human-labeled development rows to iteratively synthesize, test, and refine executable Python repair scripts with explicit applicability guards, avoiding the high cost of per-row LLM inference on the full table.

Critical strengths
  1. Code available at submission time is a great strength of the paper, especially with such an agent framework focused paper.

  2. The paper is written clearly with strong intuition guiding each of the design decisions.

  3. The experiments were thoughtfully designed with the primary results table, controlled results, API costs experiment. The paper is transparent with limitations found within the results tables.

Critical weaknesses
  1. Some of the paper is underspecified in the specific algorithmic components and framed primarily within intuition. While this is nice in terms of making the paper easier to understand and read, key details are missing. For example, a core component is the “Guarded Repair Programs”, and that in order for the strategize agent to execute the plan, it must pass guards, and an example is shown in Equation 4. However, it is unclear when or how the specific guards are used, only that they are instantiated in some way. The evidence construction is better, by specifying how the Diagnose agent classifies repair opportunities into deterministic, table-backed, or weak/external. And while the Dataset Profiler does explicitly state the cardinality, missingness, frequent values, and sampled values are used, the candidate entity-group keys and columns that are approximately constant within a group are not exactly clear what it means. The transition of initial strategy prioritizing deterministic repairs and later strategies expanding coverage intuitively makes sense, and I agree with, but it's unclear how it is operationalized. It seems like it is based on the TP, FP, FN information, but it is unclear to me exactly how. Finally, while I appreciate the representative skills in Table 1, considering that these skills are the primary mechanism that the actual cleaning is done, it would be great to understand them in detail, how they were designed, and what they are, relative to what already exists. My understanding is the cleaning skills are not novel in of themselves and novelty is derived from the overall framework, so if they were pulled from some other source that is fine, but right now, it is nebulous what they are.

  2. The comparison against baselines seems to be unfair. The first red flag is how can a fair baseline be achieving F1 scores of 0.00 on various datasets within Table 3? After I investigate it further, in Movies, TabClean can achieve 369 labeled Development Rows to “train” its approach, but the other baselines have substantially less (Baran: 20, IterClean: 5, Cocoon/AutoDCWorkflow: 0), as defined in Section 5.1. Table 4 is developed as a way to control for this for Cocoon and AutoDCWorkflow specifically, but omits Baran/IterClean, and predictably the performance increases. Please justify why this is not providing an unfair advantage and why these baseline comparisons are fair.

  3. The framework of TabClean is fairly involved, with many different subagents controlling different components. This was standard for the field a few years ago, but my impression is that we are moving more towards a unified system. The ablations somewhat show this. The Flat ablation, which combines all agents together, shows very similar performance as full on 5/6 datasets, beating full on 3/6 of them. I think further ablations should be designed which tease out the importance of each specific component and demonstrate the specific design. As stated in Weakness 1, many of the design decisions are somewhat lost and unable to be appreciated if they are careful or not.

Questions for authors

See weaknesses

Reviewer_czrkOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

Tabclean preasents an LLM-based system that synthesizes reusable Python programs for tabular data cleaning. It combines table evidence, guarded transformations, and execution feedback in a multi-agent synthesis workflow. It evaluates repair quality and inference cost across six benchmarks, including comparisons with identical development examples and test rows, and analyze benefits and limitations through ablations and sensitivity studies

Critical strengths

A key strength is its evidence-guided synthesis of repair programs with explicit guards, which grounds proposed transformations in observed data and provides a mechanism to limit unsupported changes.

Critical weaknesses

1.Although evidence-guided guard synthesis and refinement are central to the proposed approach, their specific contribution to reducing incorrect repairs is not isolated in the experiments.

2.The annotation budgets are not matched across baselines. Although Cocoon and AutoDCWorkflow are evaluated with the same labeled examples, it is unclear whether adding examples to their prompts is an appropriate adaptation of these methods. A matched-budget comparison with IterClean is also missing. While its inference cost is high, a feasible alternative would be to reduce TABClean’s annotation budget and compare both methods under the same budget.

Questions for authors

1.How were Cocoon and AutoDCWorkflow adapted to use the supplied labeled examples? Can you provide a matched-annotation-budget comparison with IterClean, potentially using a reduced TABClean budget to control inference costs?

  1. What evidence isolates the benefit of explicitly synthesizing and refining guards beyond ordinary program generation with execution feedback?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.