acceptodds
Under review as a conference paper at ICLR 2027

CleanCore: Data-Quality-Aware Coreset Selection for Learning over Dirty Tabular Data

Abstract

Multiple error types and high training costs in large-scale tabular data make it difficult to balance predictive performance and runtime efficiency. Existing data cleaning, coreset selection, and hybrid methods typically focus on one objective or specific error types. To address this, we propose **CleanCore**, a data-quality-aware coreset selection method that handles multiple types of errors. The method iteratively uses training signals for error typing and lightweight handling, and dynamically maintains training coresets based on sample reliability. Theoretical analysis provides support from three perspectives: error-typing reliability, error-handling bias, and coreset representativeness. Comparisons with eleven baselines on eleven public tabular datasets using MLP and tabular ResNet show that **CleanCore** achieves the best ACC and F1 in 21 and 20 of the 22 model–dataset settings, respectively, delivers approximately – runtime speedups over three data cleaning methods, and maintains strong performance under real-world dirty-data conditions and different injected-error settings. Further experiments validate its stability and the effectiveness and efficiency of its main mechanisms.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
coreset selection, data cleaning, robust learning, efficient training

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_prUzOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CleanCore is a coreset selection method for training deep models on large tabular datasets that contain several kinds of errors at once. It works in stages, using training signals each time. It first sorts samples by error type (label, feature or mixed), then applies a light fix to each type and adjusts each sample’s reliability weight. Finally it selects a small, representative coreset weighted by reliability. Tested against 11 baselines on 11 public datasets with MLP and tabular ResNet models, it gets the best accuracy and F1 in almost every setting. It also runs about 2–24× faster than full data-cleaning methods.

Critical strengths
  • Combining data cleaning with coreset selection is an interesting idea.
  • The method’s design makes sense at a high level.
  • End-to-end results are good, with consistently higher accuracy/F1 and lower runtime.
Critical weaknesses
  • 7 of the 11 datasets have fewer than 100K instances, which makes coreset selection unnecessary for them. More realistic, large-scale datasets (more instances and more features) would be better.
  • Standard deviations are not reported, which matters because the performance gaps are not large.
  • Several key assumptions are not verified, either theoretically or empirically. This includes the accuracy of the high-risk screening filter and of the label-consistency and feature-sensitivity checks.
  • The method has many hyperparameters. It's unclear how to set the values in practice.
Questions for authors
  • What are the precision and recall of the simple filter used for high-risk screening?
  • What are the precision and recall of the label-consistency and feature-sensitivity checks?
  • Does the coreset selection part follow or extend an existing framework? If so, it needs to be cited.
  • How should the hyperparameters be set in practice?
8 comments

Thank you for carefully reviewing our paper and for your specific suggestions on the coverage of large-scale datasets, variability in experimental results, validation of error identification, the relationship between our coreset selection and existing methods, and practical hyperparameter settings. In response, we have added experimental results and clarifications.

In the revised manuscript, substantive changes made specifically in response to your comments are marked in orange, and changes addressing comments from multiple reviewers are marked in purple. The corresponding W/Q labels appear in orange in the right margin.

To W1

Thank you for suggesting that we evaluate CleanCore on datasets with more samples and features. Our existing experiments cover a range of sample sizes, task types, and class distributions. For example, Pendigits is a relatively balanced small-scale multiclass classification dataset, Seismic Bumps is a markedly imbalanced small-scale binary classification dataset, and SUSY is a large-scale binary classification dataset with five million samples. Following your suggestion, we added a comparison on Santander, which contains 200,000 samples and 200 features, to further evaluate CleanCore in a setting with more than 100,000 samples and many features.

We selected five baselines with good predictive performance or efficiency in the main experiments, covering data cleaning, coreset selection, and hybrid methods. The paired repetitions in W2 use the same five baselines. The table below reports ACC (%), F1 (%), and TIME (s) as means ± sample standard deviations for each method over five random seeds (72–76).

Table 1. Predictive performance and runtime on Santander.

MethodACCF1TIME
CTRL86.55 ± 0.2167.90 ± 0.23462.03 ± 47.42
Cleanlab86.07 ± 0.2868.29 ± 0.24516.99 ± 43.59
CoreTab85.46 ± 0.1363.22 ± 0.6265.56 ± 4.67
GradMatch84.33 ± 0.3963.41 ± 0.1338.44 ± 4.25
Deem84.98 ± 0.8061.88 ± 0.5143.04 ± 2.21
CleanCore87.09 ± 0.1368.41 ± 0.2536.30 ± 3.01

On Santander, CleanCore achieves the highest mean ACC and F1 and the shortest mean runtime. Its mean F1 is close to that of Cleanlab, and its overall runtime is about 7% of Cleanlab's. With a runtime similar to GradMatch's, CleanCore achieves a 5.00 percentage point higher mean F1.

We have added the comparative experiment on Santander to Appendix E.8 of the revised manuscript, including the experimental setup, complete results, and analysis.

To W2

Thank you for pointing out the importance of reporting variability in experimental results. We selected MAGIC, Sensorless, MiniBooNE, and Seismic Bumps to cover different sample sizes, binary and multiclass classification tasks, and balanced and imbalanced class distributions. We used MLP as the downstream classifier and conducted paired repetitions of CleanCore and five baselines with five new random seeds (72–76). The table below reports ACC (%), F1 (%), and TIME (s) as means ± sample standard deviations.

Table 2. Predictive performance and runtime across random seeds.

DatasetMetricCTRLCleanlabCoreTabGradMatchDeemCleanCore
MAGICACC84.62 ± 0.2781.46 ± 0.3680.22 ± 0.7980.58 ± 1.2783.84 ± 0.6084.82 ± 0.19
MAGICF182.67 ± 0.2876.95 ± 0.4278.37 ± 1.0578.03 ± 0.3781.08 ± 0.8582.68 ± 0.47
MAGICTIME43.46 ± 6.0523.67 ± 6.815.04 ± 0.412.94 ± 0.273.48 ± 0.233.16 ± 0.18
SensorlessACC93.35 ± 0.1692.78 ± 0.1387.04 ± 0.6891.28 ± 0.7372.07 ± 2.3394.58 ± 0.55
SensorlessF193.21 ± 0.1992.74 ± 0.1486.96 ± 0.7391.13 ± 0.8170.91 ± 2.5894.54 ± 0.56
SensorlessTIME93.98 ± 4.48101.75 ± 4.3816.78 ± 6.5113.28 ± 2.3612.95 ± 7.299.24 ± 3.10
MiniBooNEACC89.17 ± 0.0688.91 ± 0.1887.99 ± 0.6179.18 ± 5.5985.61 ± 2.1889.43 ± 0.20
MiniBooNEF186.57 ± 0.0585.21 ± 0.1484.39 ± 1.0268.40 ± 8.5079.60 ± 4.5386.93 ± 0.21
MiniBooNETIME182.91 ± 12.10208.54 ± 7.3327.07 ± 4.5028.58 ± 12.3327.13 ± 3.8721.66 ± 3.89
Seismic BumpsACC91.45 ± 0.7591.88 ± 0.3186.81 ± 2.0091.49 ± 0.3991.95 ± 0.6292.92 ± 0.40
Seismic BumpsF152.83 ± 1.9453.89 ± 3.9652.59 ± 1.9949.33 ± 2.3652.81 ± 4.4757.80 ± 2.25
Seismic BumpsTIME17.71 ± 1.9015.31 ± 4.222.16 ± 0.392.83 ± 0.282.47 ± 0.151.77 ± 0.50

In the experiments repeated over five random seeds, CleanCore achieved the highest mean ACC and F1 among the six methods on all four datasets. On each dataset, CleanCore's sample standard deviations for ACC and F1 were lower than the average sample standard deviations of the five baselines for the corresponding metrics. These repetitions show that CleanCore maintains high classification performance on datasets of different sizes and class distributions and performs relatively consistently across random seeds. CleanCore achieved better predictive performance than the data cleaning baselines while delivering approximately 7.5–13.8× speedups. With runtime efficiency comparable to or better than that of the coreset selection baselines, CleanCore achieved better predictive performance. It combines the advantages of data cleaning and coreset selection and improves both predictive performance and runtime efficiency.

We have added the setup and analysis of the paired repetitions to Appendix E.6 of the revised manuscript and report the mean and sample standard deviation of each metric in Table 14.

To W3, Q1, and Q2

Thank you for pointing out that error identification needs further validation. Following your suggestion, we use loss and gradient signals from training to explain the key assumptions underlying high-risk screening and error typing and the rationale for them. We also use the detection records from the repeated experiments in W2 to evaluate how high-risk screening and error typing perform in practice.

Approximate consistency of training updates. Assumption 1 first requires that actual training and ideal clean training start from the same model and have sufficiently close parameter updates within the current sliding window. MisDetect (Deng et al., 2024) observes that models tend to fit clean samples first during early training. This supports the premise that early updates in actual training can still follow the classification patterns in the clean data. In later stages, CleanCore progressively corrects identified errors, down-weights suspicious samples, and prioritizes reliable samples as coreset representatives. This reduces the interference of corruption with subsequent updates, making the actual updates more likely to be close to those in ideal clean training.

Correctness of ideal error typing. Assumption 1 also requires that, for samples identified as erroneous at the current stage, the signals from ideal clean training yield the correct type under the same decision rules and thresholds. The rationale is that the relationships between features and labels learned by the ideal model from clean data can help identify the source of a high loss. If the features still carry information about the true class, replacing the label with one consistent with those features may substantially reduce the loss and bring it close to normal levels. If the abnormal loss mainly stems from a few corrupted features, it may be more sensitive to those features. Keeping the label fixed and making a small adjustment to those features may also reduce the loss.

The above analysis explains the rationale for Assumption 1. CleanCore aims to reduce the interference of data corruption with model training and improve predictive performance. Error typing guides subsequent handling without requiring identification of the true error type of every corrupted sample. As described in the paper, samples retained in the training pool can be checked again in later stages. CleanCore can therefore focus on accurately typing detected samples without having to detect every error in a single stage, reducing the risk of incorrectly modifying correct labels or features. To evaluate actual screening and typing performance, we compare the full-run detection records for the four datasets and five random seeds in W2 against the known injected error types, sample by sample. Each sample is counted once per seed, and the counts are pooled across the five seeds to calculate precision and recall.

Reviewer_V1cGOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CleanCore jointly handles dirty tabular data and selects training coresets. It uses recent loss statistics, virtual label replacement, and feature gradients to assign candidate error types; applies soft relabeling, feature repair, or reliability downweighting; and maintains a weighted gradient-coverage coreset through rebuilding and local updates. Experiments compare eleven baselines on eleven datasets with MLP and tabular ResNet, including injected errors and a real-world dirty-data benchmark. The paper reports leading accuracy and macro-F1 in most settings, lower runtime than cleaning methods, component ablations, and conditional analyses of typing, handling, and coreset approximation.

Critical strengths
  1. The work addresses a practical interaction between data quality and training cost. Reliability weights affect both error handling and representative selection, and adaptive maintenance avoids rebuilding every stage. The evaluation includes cleaning, selection, hybrid, and sequential baselines, making the joint formulation more informative than a comparison against only one category.
  2. The empirical coverage is broad: eleven datasets, two neural architectures, several noise levels and types, and native dirty SmartFactory data. Tables 1 and 3 report strong predictive results with total runtime, rather than training time alone. The No-FER and Static-Core controls in Appendix E.4 provide useful evidence that feature repair and changing representatives contribute beyond reliability weighting alone.
Critical weaknesses
  1. The error-typing rules do not yet convincingly distinguish corrupted labels from corrupted features or hard clean examples. The label-consistency check runs first: a feature-corrupted example that makes the model confidently predict the wrong class can also show a large loss reduction after label replacement, routing it to label softening before feature checking. Assumption 1 explicitly assumes correctness of ideal typing, so Theorem 1 establishes conditional stability rather than resolving this ambiguity. Appendix E.5 reports correctness among typed candidates, but not a per-type confusion matrix, detection recall, or false positives on hard clean/minority examples. This matters because incorrect repairs can reinforce the current model's errors.
  2. The empirical conclusions are not sufficiently reproducible or statistically characterized. Appendix E.1 states that all experiments use the same random seed, and the main comparisons provide no variation across splits, corruption draws, or initializations. Some wins over the nearest baseline are small. The manuscript also leaves important defaults at the level of principles, including corruption magnitudes/type proportions, several threshold quantiles and cutoffs, and model widths/stage settings. The threshold sweeps are useful but do not provide a complete runnable configuration, limiting confidence in the claimed broad advantage.
Questions for authors
  1. Can you provide a confusion matrix for clean, label, feature, and mixed cases, including recall and the false-positive rate on hard clean/minority examples? In particular, what happens to feature-only corruptions that produce confident wrong predictions at the label-consistency step? Please distinguish the conditional theorem from an identification guarantee.
  2. Can you supply complete per-dataset configurations and repeated paired results on representative small, imbalanced, and large datasets? Please include split/corruption/initialization variability, exact corruption settings, architecture sizes, stage schedule, threshold defaults, and the corresponding runtime variation. These would materially strengthen the broad comparison.
11 comments

Thank you for your careful review of our paper and for your specific suggestions on the reliability of error typing, the conditions under which the theoretical results apply, the experimental setup, and variability in the results. In response, we have added clarifications and experimental results.

In the revised manuscript, substantive changes made specifically in response to your comments are marked in blue, and changes addressing comments from multiple reviewers are marked in purple. The corresponding W/Q labels appear in blue in the right margin.

To W1 and Q1

Thank you for carefully examining our error-typing rules. Regarding the high-confidence misclassification scenario you raised, we further clarify how the label-consistency check classifies such samples and how CleanCore then handles them. Following your suggestions, we also clarify the assumptions underlying Theorem 1 and their rationale, and report full-run error-typing results and false positive rates across five random seeds.

(1) Handling samples with pure feature corruption that are misclassified with high confidence

For the samples you describe, suppose a sample passes high-risk screening. If virtually replacing its label with the class predicted by the model with high confidence yields a relative reduction in window-mean loss that reaches the threshold and a mean loss after replacement within its upper bound, CleanCore classifies the sample as a label error at that stage and sends it to the label-error softening branch.

This can happen because pure feature corruption sometimes makes a sample exhibit characteristics typical of another class, so the model is more likely to predict that class with high confidence. Given only the corrupted features and the original label, a person inspecting the sample might also suspect a labeling error. In the label-error softening branch, CleanCore increases the probability assigned in the soft label to the candidate class that passes the check. Even if the decision at this stage disagrees with the true error type, the corrupted features are still consistent with that class. The soft label can provide effective supervision for that class while mitigating training interference from the mismatch between the original label and the current features. If soft-label confidence remains low, CleanCore reduces the training weight of the sample to limit the impact of potential mistyping on model updates.

CleanCore aims to reduce the interference of data corruption with model training and improve predictive performance. Error typing guides sample handling without requiring identification of the true error type of every corrupted sample.

(2) Assumptions underlying Theorem 1 and their rationale

Approximate consistency of training updates. Assumption 1 first requires that actual training and ideal clean training start from the same model and have sufficiently close parameter updates within the current sliding window. MisDetect (Deng et al., 2024) observes that models tend to fit clean samples first during early training. This supports the premise that early updates in actual training can still follow the classification patterns in the clean data. In later stages, CleanCore progressively corrects identified errors, down-weights suspicious samples, and prioritizes reliable samples as coreset representatives. This reduces the interference of corruption with subsequent updates, making the actual updates more likely to be close to those in ideal clean training.

Correctness of ideal error typing. Assumption 1 also requires that, for samples identified as erroneous at the current stage, the signals from ideal clean training yield the correct type under the same decision rules and thresholds. The rationale is that the relationships between features and labels learned by the ideal model from clean data can help identify the source of a high loss. If the features still carry information about the true class, replacing the label with one consistent with those features may substantially reduce the loss and bring it close to normal levels. If the abnormal loss mainly stems from a few corrupted features, it may be more sensitive to those features. Keeping the label fixed and making a small adjustment to those features may also reduce the loss.

To W1 and Q1 (continued)

(3) Experimental results

To evaluate how CleanCore types errors in actual training, we compare the full-run detection records from the five random seeds in W2 against the known injected error types, sample by sample.

Full-run error-typing confusion matrices

Each training sample is counted once per seed. Samples that pass only the label check or only the feature check over the entire run are classified as label-only errors or feature-only errors, respectively. Samples that pass the two checks in different stages, or pass neither check over the entire run but enter the mixed-error branch at least once, are classified as mixed errors. Samples that never enter any error branch over the entire run are recorded as not detected. Assignment to the mixed-error branch indicates that the error type is uncertain at that stage. The full-run classification therefore gives priority to the checks the sample has passed.

Table 1. Error-typing confusion matrices over the full training run.

DatasetTrue typeNot detectedLabel errorFeature errorMixed errorTotal
MAGICClean60486975222560,860
MAGICLabel only23173489382416,085
MAGICFeature only274621724826406,085
MAGICMixed9761498718333,045
SensorlessClean185982163112963187,220
SensorlessLabel only58499725482266418,720
SensorlessFeature only85307268192127218,720
SensorlessMixed184638615569789,365
MiniBooNEClean4083639919725879416,205
MiniBooNELabel only14798193442262521641,620
MiniBooNEFeature only20939169316792219641,620
MiniBooNEMixed89871652825934620,810
Seismic BumpsClean8173187728,270
Seismic BumpsLabel only2125262859825
Seismic BumpsFeature only3363439857825
Seismic BumpsMixed873213283415

To W1 and Q1 (continued)

Recall for the four sample types

Table 2 reports recall (%) for the four types over all training samples. Both the label-consistency check and the feature-sensitivity check follow high-risk screening. To assess typing on the actual candidates, we also compute full-run recall for label-only and feature-only errors among samples screened as high risk at least once. The analysis below reports these results.

Table 2. Recall by sample type.

DatasetCleanLabel-only errorFeature-only errorMixed error
MAGIC99.3957.3440.7960.20
Sensorless99.3451.9543.7674.51
MiniBooNE98.1246.4840.3544.91
Seismic Bumps98.8363.7648.2468.19

False positive rates

For the difficult clean samples you asked about, training difficulty depends on the model and its training state, making it difficult to assign difficulty labels consistently before training. Table 3 therefore reports false positive rates (%) for all clean samples and separately for clean minority-class samples on binary classification datasets. We determine the minority class from the true label counts in the training set before error injection. All 11 classes in Sensorless have the same number of samples, so the minority-class false positive rate does not apply. Both rates use counts pooled over the five seeds.

Table 3. False positive rates for clean samples.

DatasetAll clean FPRClean minority FPR
MAGIC0.610.73
Sensorless0.66-
MiniBooNE1.881.46
Seismic Bumps1.171.13

The false positive rate for all clean samples is 0.61%–1.88% across the four datasets, and that for clean minority-class samples is 0.73%–1.46% across the three binary classification datasets. Among samples screened as high risk at least once, full-run precision is 81.69%–88.41% for label-only errors and 80.53%–93.34% for feature-only errors. The corresponding recall ranges are 72.12%–92.60% and 74.33%–81.39%. CleanCore aims to improve predictive performance and places greater emphasis on accurately typing detected samples. The three tables show that CleanCore flags few clean samples and correctly identifies most screened label-only and feature-only errors, providing a basis for subsequent label softening and feature repair.

Across all four datasets, more than half of the detected but incorrectly typed corrupted samples are ultimately classified as mixed errors. In a given stage, when neither check passes and the error type remains uncertain, CleanCore temporarily assigns the sample to the mixed-error branch and reduces its training weight while keeping its label and features unchanged. This reduces the influence of suspicious samples on model updates and the risk of incorrectly modifying their labels or features due to a typing error.

Appendix D.1 of the revised manuscript describes how samples misclassified with high confidence are handled, as well as the assumptions and conditions of Theorem 1. Appendix E.5 (1) provides the full-run error-typing results, counting protocol, and false positive rates.

Reviewer_nqYiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper proposes CleanCore, a framework for efficient training of neural classifiers on tabular data containing label errors, feature errors, and combinations of both. After initial training on the full dataset, it alternates between three components: ErrorTyping, ErrorHandle, and CoreSelect.

The main claimed contribution is the integration of differentiated error handling with dynamic coreset maintenance. The paper evaluates 11 datasets, 11 main baselines, and 2 neural architectures (MLP, tabular ResNet). The authors claim best-or-tied-best accuracy in 21 of 22 model-dataset settings and Macro-F1 in 20 of 22 settings, and 2 to 24 x runtime improvements over the baselines.

The evidence currently supports the effectiveness of the combined pipeline more convincingly than it establishes reliable error identification, the particular contribution of the selector, or preservation of clean full-data training performance.

Critical strengths

S1.S1. The problem is important and the proposed combination of data cleaning with coreset selection is interesting

S2.S2. The benchmark coverage is substantial

Critical weaknesses

W1.W1. The fixed-seed evaluation does not establish reliable superiority over the strongest baselines

Appendix E.1 states that all experiments use the same random seed. Multiple independent runs, variability across corruption draws, and uncertainty intervals are not reported. Several of the leading margins are very small therefore the importance of multiple runs. The claim of being best in almost every setting needs repeated-seed evidence.

W2.W2. Important implementation and evaluation choices remain insufficiently specified

The appendix leaves several choices needed for reproduction unspecified. These include:

  • Network depths and widths.
  • Dataset-specific batch sizes and maximum epoch counts.
  • Early-stopping tolerance and patience.
  • The main-run values of numerous thresholds and handling parameters.
  • The distributions and magnitudes of numerical corruptions, the allocation among label/feature/mixed errors, and the number of affected features.
  • Baseline-specific settings and the hyperparameter-selection protocol.

Appendix D.4 explains threshold-setting principles, but does not provide a complete configuration for reproducing the reported results.

Appendix E.1 similarly describes numerical perturbations and distribution shifts without specifying their parameters. The reproducibility statement promises code after acceptance.

These omissions also affect interpretation of the runtime comparisons. Training stops when training loss ceases to improve sufficiently, but cleaning, relabeling, reweighting, and pruning change that loss and the examples contributing to it. Consequently, shared stopping rules can still produce different stopping behavior. Reporting completed epochs, optimization steps, stage/rebuild counts, and predictive performance against elapsed time would make the speedup claim easier to assess.

W3.W3. ErrorTyping has a substantive ambiguity between label errors, feature errors, and model mistakes

The algorithm tests label replacement before feature sensitivity. If another label sufficiently reduces the loss, the example is classified as a label error and does not reach the feature check. (Section 3.1, pp. 4-5; Appendix D.1, pp. 20-21.)

A feature corruption that moves an example into another class's confident prediction region can therefore appear to be a label error.

Figure 12(a) reports high typing correctness among selected candidates, which is useful evidence. However, it does not provide the number of selected examples, recall by corruption type, a confusion matrix including clean examples, or false-positive/removal rates for hard and minority-class examples. Those measurements are particularly relevant to the claimed protection of difficult but correct data. (Appendix E.5, p. 46.)

W4.W4. The theory provides conditional support, but does not establish the difficult properties of the method

  • Theorem 1 (typing reliability) has a main limitation in that Assumption 1 already assumes that the ideal clean-training signals give correct error-type judgments.
  • Theorem 2 (handling bias)'s main limitation: Contraction for label and feature handling is assumed, rather than derived from the proposed update rules.
  • Theorem 3 (coreset representativeness)'s main limitation: The exact greedy guarantee does not cover the sampled-batch implementation or establish closeness to full clean-data training.

Theorem 2 additionally concerns identified erroneous samples retained after handling, and compares processed and clean-reference gradients using the same reliability weights. It therefore does not control all the effects of removing examples, reducing the weights of clean examples, or leaving corruption undetected. These effects matter for the clean full-data objective in Definition 1.

The empirical handling diagnostic also needs careful interpretation. Figure 12(b) reports how often the weighted gradient residual decreases. Down-weighting alone can improve this metric; for the mixed-error branch, that reduction follows directly from the update rule. Furthermore, the proportion of individually improved examples does not establish that the aggregate discrepancy decreases, since a small number of large failures could dominate. (Appendix E.5, p. 46)

If the proofs appear plausible under their stated assumptions, their limited scope means that stronger empirical validation must carry most of the paper's argument.

W5.W5. Essential reference controls and comparisons that isolate the new contribution are missing

Definition 1 frames the goal as approaching the performance of full clean-data training while reducing computation. However, the reported comparisons do not include ordinary training on the full dirty dataset or full-data training on the available clean reference datasets. These are important anchors for assessing the proposed objective. (Definition 1, p. 3; Tables 1, 3, and 6.)

Table 6 compares the complete CleanCore pipeline with random selection. It does not isolate the benefit of CoreSelect over random selection when both receive the same error handling and reliability estimates. A matched comparison would help determine how much of the improvement comes from processing the data versus selecting representatives.

The novelty should also be positioned more precisely, for example:

  • Deem (Deng et al.,2025) already combines training losses and gradients, soft-label gradient approximation, and dynamic subset refinement.
  • CRAIG (Mirzasoleiman et al., ICML 2020) already uses nearest-representative gradient coverage, representative weights, and a facility-location formulation.
  • CRUST (Mirzasoleiman et al., NeurIPS 2020) directly studies robust training through iterative gradient-space coreset selection under noisy labels, making it a relevant comparison absent from the current evaluation.

CleanCore's multi-error treatment and reliability-driven maintenance could constitute a useful contribution. A controlled comparison that holds those treatments fixed while changing the selector would establish that contribution more clearly. Separate label-only, feature-only, and joint-corruption results would also clarify where the extension provides its benefit.

W6.W6. The "real-world dirty data" claim needs clearer provenance

SmartFactory is the sole experiment presented as evidence for naturally occurring dirty data, and Appendix E.4 calls it "native dirty data". Yet the cited REIN benchmark describes injecting errors into real-world datasets to obtain controlled dirty/clean pairs; its Table 4 lists Smart Factory among those benchmark datasets.

Questions for authors
  1. Do the improvements over the strongest baselines persist across independent training seeds and corruption realizations?

  2. How the hyperparameters are set in practice?

  3. How does the label-first rule distinguish a genuine label error from a feature-corrupted or clean difficult example that the model confidently predicts as another class?

  4. What properties of the actual update rules support the ideal-typing and gradient-contraction assumptions in Theorems 1 and 2? Can you report aggregate clean-reference gradient discrepancies and separate correction effects from the reduction obtained by down-weighting alone?

  5. How does CleanCore compare with full (real-world) dirty-data training, full clean-reference training, and a variant that retains the same error handling/reliability weights but uses random or an established gradient-based selector? Please compare predictive performance against elapsed time under an explicitly specified stopping and model-selection protocol.

11 comments

Thank you for your careful review of our paper and for your specific suggestions, including those on the experimental setup, theoretical analysis, and comparative evaluation. In response, we have added experimental results and clarifications.

In the revised manuscript, substantive changes made specifically in response to your comments are marked in red, and changes addressing comments from multiple reviewers are marked in purple. The corresponding W/Q labels appear in red in the right margin.

To W1 and Q1

Thank you for pointing out the limitation of evaluation with a fixed random seed. To test whether CleanCore's gains over the strongest baseline in our comparison persist across independent training seeds and different error-injection instances, we selected Magic, Sensorless, MiniBooNE, and Seismic Bumps. These datasets span different sample sizes, binary and multiclass classification tasks, and balanced and imbalanced class distributions. We selected five baselines from the main experiments based on their predictive performance or efficiency. They cover data cleaning (CTRL and Cleanlab), coreset selection (CoreTab and GradMatch), and hybrid methods (Deem).

We use MLP as the downstream classifier and conduct paired repetitions with five new random seeds (72–76). For each dataset, the five runs use different error-injection instances. Within each seed, all methods share the training/test split, the error-injection instance, and model initialization. The table below reports ACC (%), F1 (%), and TIME (s) as mean ± sample standard deviation.

Table 1. Predictive performance and runtime.

DatasetMetricCTRLCleanlabCoreTabGradMatchDeemCleanCore
MagicACC84.62 ± 0.2781.46 ± 0.3680.22 ± 0.7980.58 ± 1.2783.84 ± 0.6084.82 ± 0.19
MagicF182.67 ± 0.2876.95 ± 0.4278.37 ± 1.0578.03 ± 0.3781.08 ± 0.8582.68 ± 0.47
MagicTIME43.46 ± 6.0523.67 ± 6.815.04 ± 0.412.94 ± 0.273.48 ± 0.233.16 ± 0.18
SensorlessACC93.35 ± 0.1692.78 ± 0.1387.04 ± 0.6891.28 ± 0.7372.07 ± 2.3394.58 ± 0.55
SensorlessF193.21 ± 0.1992.74 ± 0.1486.96 ± 0.7391.13 ± 0.8170.91 ± 2.5894.54 ± 0.56
SensorlessTIME93.98 ± 4.48101.75 ± 4.3816.78 ± 6.5113.28 ± 2.3612.95 ± 7.299.24 ± 3.10
MiniBooNEACC89.17 ± 0.0688.91 ± 0.1887.99 ± 0.6179.18 ± 5.5985.61 ± 2.1889.43 ± 0.20
MiniBooNEF186.57 ± 0.0585.21 ± 0.1484.39 ± 1.0268.40 ± 8.5079.60 ± 4.5386.93 ± 0.21
MiniBooNETIME182.91 ± 12.10208.54 ± 7.3327.07 ± 4.5028.58 ± 12.3327.13 ± 3.8721.66 ± 3.89
Seismic BumpsACC91.45 ± 0.7591.88 ± 0.3186.81 ± 2.0091.49 ± 0.3991.95 ± 0.6292.92 ± 0.40
Seismic BumpsF152.83 ± 1.9453.89 ± 3.9652.59 ± 1.9949.33 ± 2.3652.81 ± 4.4757.80 ± 2.25
Seismic BumpsTIME17.71 ± 1.9015.31 ± 4.222.16 ± 0.392.83 ± 0.282.47 ± 0.151.77 ± 0.50

In the five paired experiments with different training seeds and error-injection instances, CleanCore achieves the highest mean ACC and F1 on all four datasets. For both metrics on each dataset, its sample standard deviation is lower than the average sample standard deviation of the five baselines. Compared with the best-performing data cleaning baseline, CleanCore has a slightly higher mean F1 than CTRL on Magic and improves mean F1 by 0.36–3.91 percentage points on the other three datasets. CleanCore achieves approximately 7.5–13.8× speedups over the two data cleaning baselines. Compared with the best-performing baseline among CoreTab, GradMatch, and Deem, CleanCore improves mean F1 by 1.60–4.99 percentage points and has comparable or shorter runtimes. Overall, CleanCore maintains its mean predictive performance advantage over the strongest baseline in the comparison while remaining runtime-efficient. We have added the experimental setup, complete results, and analysis to Appendix E.6.

To W2 and Q2

Thank you for specifically pointing out the settings that affect experimental reproducibility and runtime comparisons. In response to your comments, we provide the configurations used in our experiments and the rules for selecting baseline configurations. We also report CleanCore's actual training workload and how its predictive performance changes over time.

(1) Experimental configurations

Dataset settings

The training/test splits follow Appendix E.1. We corrupt 20% of the training samples on the ten datasets with clean references. Label, feature, and mixed errors are assigned in a 2:2:1 ratio, accounting for 8%, 8%, and 4% of the training set, respectively. SmartFactory uses its existing dirty training set. For samples involving feature errors, we modify at least one feature and at most approximately 30% of the features.

For random numerical errors, 80% of selected numerical cells are randomly replaced with other valid values, 10% are set to missing, and the remaining 10% are replaced with randomly generated out-of-range values. For numerical distribution shifts, we randomly choose a fixed direction for each numerical feature and randomly draw the shift magnitude between one and two times that feature's standard deviation in the training set.

Architectures and training settings

MLP has two hidden layers of widths 256 and 128. Each uses BatchNorm, ReLU, and dropout at a rate of 0.20. A linear classifier outputs the class scores, bringing the total number of linear layers to three. Tabular ResNet first projects the input to 64 dimensions and then applies two residual blocks. Each block uses two linear transformations of dimensions 64 → 128 → 64, with LayerNorm, ReLU, and dropout at a rate of 0.20. A final LayerNorm and ReLU precede the linear classifier, bringing the total number of linear layers to six. Following Gorishniy et al. (NeurIPS 2021), we use MLP and tabular ResNet as classifiers for tabular data. CleanCore achieves efficiency gains with both architectures. For models with higher per-step training costs, reducing the number of training samples accessed could save more training time.

Both classifiers use the same training settings. We use AdamW with a learning rate of 0.001 and weight decay of 0.0001. The maximum number of training epochs depends on the training set size. It is 400 for Covertype and SUSY and 500 for all other datasets. Early stopping monitors training loss with a minimum improvement of 0.0001. Patience is 40 epochs for Covertype and SUSY and 50 for all other datasets. Whether training reaches the epoch limit or triggers early stopping, each method evaluates the model from the final completed epoch on the test set.

Table 2. Batch sizes by dataset.

DatasetShared batch size
WDBC, Banknote, Seismic Bumps64
Pendigits256
Magic512
SmartFactory, Sensorless1024
MiniBooNE2048
Skin2048
Covertype4096
SUSY8192

CleanCore settings

CleanCore first pretrains on the full training set for five epochs and then performs a stage update every Etr=35E_{\mathrm{tr}}=35 epochs. The sliding window has length L=5L=5 and uses the five most recent snapshots. The coreset budget KK depends on the training set size. It is 10% for fewer than 25,000 samples, 5% from 25,000 to fewer than 1,000,000 samples, and 1% otherwise.

To W2 and Q2 (continued)

For high-risk screening, the quantile levels for the mean-loss threshold τμ(t)\tau_\mu^{(t)} and loss-standard-deviation threshold τσ(t)\tau_\sigma^{(t)} are P70 and P30, respectively, and the high-value threshold coefficient is κ=0.50\kappa=0.50. For label-error identification, the minimum relative loss reduction τrel(t)\tau_{\mathrm{rel}}^{(t)} is 30%, and the replacement-loss upper bound τclean(t)\tau_{\mathrm{clean}}^{(t)} is P40 of the current stage's sample mean-loss distribution. Among the remaining candidates, the quantile level for the gradient-magnitude threshold τA(t)\tau_A^{(t)} is P60. For dd input features, the maximum number inspected is m0=max⁡{1,⌊d/5⌋}m_0=\max\lbrace 1,\lfloor d/5\rfloor\rbrace. A virtual update is performed only if the cumulative gradient proportion of at most the m0m_0 highest-gradient-magnitude features reaches τconc(t)=0.50\tau_{\mathrm{conc}}^{(t)}=0.50. The virtual step size is ηfeat=0.05\eta_{\mathrm{feat}}=0.05, and the minimum relative loss reduction τfeat(t)\tau_{\mathrm{feat}}^{(t)} is 2%. Quantiles are computed on the corresponding sample sets at the current stage. High-value thresholds take the larger of the corresponding quantile and the signal mean plus 0.50 times its standard deviation. Label- and feature-error identification use relative loss reduction, the difference in mean loss before and after the virtual change divided by the mean loss before the change. The initial step size for actual feature repair uses the absolute loss reduction, the mean loss before the virtual update minus the mean loss after it. In the revised manuscript, we clarify the relative loss reduction criteria in Section 3.1 and Appendix D.1 and explain the use of absolute loss reduction to calculate the initial step size in Appendix D.2.

The soft-label temperature coefficient is α=2.5\alpha=2.5, the weight on the current judgment is β=0.70\beta=0.70, and the soft-label confidence down-weighting coefficient is λ=0.5\lambda=0.5. A soft label is converted to a hard label when the confidence of the same dominant class reaches 0.90 in two consecutive stages. Actual feature repair uses a base scaling factor ηbase=1.0\eta_{\mathrm{base}}=1.0, a step-size coefficient cap ηmax⁡=0.05\eta_{\max}=0.05, a target loss reduction coefficient λtar=0.45\lambda_{\mathrm{tar}}=0.45, and at most Mfeat=3M_{\mathrm{feat}}=3 repair steps. The post-repair weight smoothing coefficient is ηw=0.50\eta_w=0.50. Mixed-error sample weights are multiplied by λmix=0.45\lambda_{\mathrm{mix}}=0.45. The weight of a sample not identified as erroneous increases by ηclean=0.05\eta_{\mathrm{clean}}=0.05 at each stage, up to 1. A sample is removed if it is still high risk and its weight falls below wmin⁡=0.05w_{\min}=0.05.

For coreset maintenance, the representative-candidate reliability threshold is τrep=0.50\tau_{\mathrm{rep}}=0.50. In the corresponding coreset maintenance branches, the sample remapping threshold τmap\tau_{\mathrm{map}} and cluster stability threshold τclu\tau_{\mathrm{clu}} determine which samples and clusters require maintenance. After initial coreset construction, these thresholds are set to P70 of the weighted gradient deviations of nonrepresentatives from their respective representatives and P70 of the within-cluster weighted mean gradient deviations across clusters, respectively. Both thresholds are held fixed in later stages. The representative-cluster check threshold τupd\tau_{\mathrm{upd}} and the rebuilding threshold τreb\tau_{\mathrm{reb}} depend on the training set size. For training sets with fewer than 250,000 samples, τupd=0.30\tau_{\mathrm{upd}}=0.30 and τreb=0.75\tau_{\mathrm{reb}}=0.75. For training sets with at least 250,000 samples (Covertype and SUSY), they are τupd=0.45\tau_{\mathrm{upd}}=0.45 and τreb=0.90\tau_{\mathrm{reb}}=0.90. The higher thresholds reduce the cost of representative-cluster checks and coreset rebuilding on large datasets, while rebuilding is triggered when the approximation error exceeds the rebuilding threshold to maintain representativeness.

Baseline configurations and selection procedure

On the same dataset and with the same downstream model, all methods share the data split, error-injection instance, preprocessing, common training settings, and random seed. We compare both predictive performance and runtime. Runtime covers the cleaning, sample selection, and model training performed by each method. Data cleaning methods still train a classifier after cleaning. On large datasets, the runtime of some methods with default configurations substantially exceeds the comparison budget. We therefore set a common time limit for the baselines using the measured runtime of a single Random run on the same dataset with the same downstream model and coreset budget as the reference. Table 13 of the revised manuscript reports the Random runtimes. The time limit is 40 times the reference runtime for training sets with fewer than 1,000,000 samples and 160 times for training sets with at least 1,000,000 samples.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for taking the time to carefully review our paper and read our responses, and for making specific suggestions and asking follow-up questions during the discussion. Your feedback helped us identify which conclusions needed more experimental support and gave us the opportunity to explain CleanCore’s key mechanisms more clearly. We took the feedback from all reviewers into account and added experiments and clarifications in response. Below are the main additions to the revised manuscript to date.

  1. Paired repetitions and training costs. We added paired repetitions across multiple random seeds and reported the actual training workload, the number of coreset rebuilds, and performance under different training budgets. (Appendix E.6)

  2. Experimental settings and tuning. We added experimental configurations, the procedure for selecting baseline configurations, and practical tuning guidance. (Appendices D.4(2) and E.1)

  3. Validation of high-risk screening and error typing. We added an evaluation covering the entire training run and reported precision and recall for screening and error typing, as well as false positive and removal rates for clean samples. (Appendix E.5(1))

  4. High-confidence misclassification and hard-label conversion. We further explained how CleanCore handles feature-corrupted samples that are misclassified with high confidence, clarified cross-stage reassessment and label updates after hard-label conversion, and discussed cases where a sample is not flagged again because its loss decreases. (Appendices D.1 and D.2)

  5. Theoretical assumptions and gradient deviation. We further explained the rationale for the assumptions underlying the theorems and experimentally evaluated the effects of correction and down-weighting on full-pool gradient deviation. (Appendices D.1–D.3 and E.5(2))

  6. CoreSelect and additional controls. We further clarified how CoreSelect relates to existing gradient-based coreset methods. We added comparisons with full-data training, a random selector, and CRUST, as well as experiments under label-only and feature-only corruption. (Section 3.3 and Appendices D.4(1) and E.7)

  7. Santander evaluation and SmartFactory data sources. We added an evaluation on Santander, a dataset with a relatively large number of samples and features, and clarified the sources of the SmartFactory data and its corruption. (Section 4.1 and Appendices E.1 and E.8)

The revised manuscript summarizes the main new results in the main text and provides the complete experimental settings, results, and methodological explanations in the relevant sections and appendices. We use colors and margin labels to indicate which reviewer comments the revisions address. We again thank all reviewers for the feedback they provided during the review and discussion, which helped us further improve the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.