- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DIDA, a value driven tabular data augmentation framework for low data, high constraint scenarios. DIDA uses SHAP to estimate feature level importance and KNN Shapley to estimate sample level contribution, and uses these two signals to guide enhancement of sample feature levels. This method also introduces dynamic domain correction to preserve high-value boundary samples, constructs constraint tables from cross feature associations to guide classification enhancement, and applies value related perturbations to numerical features.
Critical strengths
1.DIDA targets a practically important low data, high constraint setting, in which traditional enhancement methods are prone to violate feature dependency relationships or distort information samples. Therefore, its value consciousness and constraint oriented design are very suitable for such applications.
2.The proposed enhancement mechanism is relatively interpretable and auditable. DIDA utilizes interpretable methods such as feature level importance and sample level contribution for generation, allowing enhanced decisions to be traced back to identifiable value scores and constraint rules, rather than opaque generation processes. This is particularly useful in regulated or high-risk spreadsheet applications, where understanding the reasons for sample modifications is equally important as predictive performance.
Critical weaknesses
1.DIDA uses high contribution samples to expand various feature domains . If a high-value sample violates substantive business rules rather than just statistical boundaries, this correction step may make previously invalid values "valid" and be included in the category of data augmentation. Furthermore, even if the range of values for a single feature is met, it may not necessarily ensure that the generated combination of numerical features conforms to the real-world business constraints across features. If the difference between "statistical outliers" and "violation of strict business rules" can be more clearly defined, and constraint verification can be clearly performed after correction, it will help improve the security and applicability of this method in strong constraint application scenarios.
2.When the precise constraint table lookup fails, DIDA gradually relaxes the conditional constraints, and it is unclear whether the final complete feature combination still meets the original constraints. The current ablation mainly evaluates downstream predictive performance, rather than directly reporting constraint violation rates (CVR) at different rollback levels. Direct evaluation of invalid or unsupported combinations, especially after relaxation, margin, and uniform rollback, is crucial for verifying the claimed business compliance assurance.
Questions for authors
See Weaknesses
6 comments
Table: ROC-AUC of each method at different training fractions (mean ± sample standard deviation)
| Dataset | Train (%) | Original | SMOTE | CTGAN | SMOTE-CDNN | TabDDPM | MDLM-local | TANDEM | DIDA |
|---|---|---|---|---|---|---|---|---|---|
| Alzheimers | 25% | <u>0.8224 ± 0.0075</u> | 0.8183 ± 0.0088 | 0.7870 ± 0.0123 | 0.8171 ± 0.0038 | 0.7901 ± 0.0097 | 0.8077 ± 0.0133 | 0.8107 ± 0.0088 | 0.8429 ± 0.0094 |
| 50% | 0.8507 ± 0.0096 | 0.8517 ± 0.0121 | 0.8314 ± 0.0075 | <u>0.8535 ± 0.0091</u> | 0.8301 ± 0.0108 | 0.8377 ± 0.0084 | 0.8378 ± 0.0059 | 0.8654 ± 0.0080 | |
| 75% | <u>0.8630 ± 0.0078</u> | 0.8617 ± 0.0085 | 0.8497 ± 0.0110 | 0.8606 ± 0.0099 | 0.8447 ± 0.0082 | 0.8514 ± 0.0080 | 0.8529 ± 0.0069 | 0.8708 ± 0.0069 | |
| 100% | <u>0.8672 ± 0.0076</u> | 0.8654 ± 0.0104 | 0.8538 ± 0.0068 | 0.8656 ± 0.0075 | 0.8508 ± 0.0101 | 0.8580 ± 0.0084 | 0.8577 ± 0.0091 | 0.8745 ± 0.0062 | |
| Kidney | 25% | 0.6332 ± 0.0313 | 0.6398 ± 0.0295 | 0.6150 ± 0.0385 | 0.6434 ± 0.0315 | 0.5933 ± 0.0341 | 0.6434 ± 0.0247 | <u>0.6457 ± 0.0348</u> | 0.6639 ± 0.0418 |
| 50% | 0.6765 ± 0.0311 | 0.6776 ± 0.0271 | 0.6495 ± 0.0347 | <u>0.6837 ± 0.0280</u> | 0.6244 ± 0.0341 | 0.6720 ± 0.0285 | 0.6741 ± 0.0205 | 0.6945 ± 0.0246 | |
| 75% | 0.6939 ± 0.0252 | <u>0.6998 ± 0.0299</u> | 0.6609 ± 0.0344 | 0.6974 ± 0.0300 | 0.6471 ± 0.0342 | 0.6930 ± 0.0291 | 0.6878 ± 0.0402 | 0.7099 ± 0.0281 | |
| 100% | 0.7050 ± 0.0167 | 0.7100 ± 0.0257 | 0.6770 ± 0.0242 | 0.7149 ± 0.0190 | 0.6589 ± 0.0305 | 0.7129 ± 0.0338 | 0.6939 ± 0.0172 | <u>0.7136 ± 0.0187</u> | |
| Heart | 25% | 0.8203 ± 0.0153 | 0.8127 ± 0.0099 | 0.8131 ± 0.0210 | 0.8157 ± 0.0147 | 0.8261 ± 0.0183 | 0.8201 ± 0.0141 | <u>0.8270 ± 0.0196</u> | 0.8306 ± 0.0170 |
| 50% | 0.8392 ± 0.0171 | 0.8262 ± 0.0226 | 0.8277 ± 0.0187 | 0.8324 ± 0.0227 | <u>0.8398 ± 0.0161</u> | 0.8368 ± 0.0219 | 0.8328 ± 0.0201 | 0.8489 ± 0.0236 | |
| 75% | 0.8453 ± 0.0202 | 0.8397 ± 0.0234 | 0.8439 ± 0.0267 | 0.8393 ± 0.0206 | <u>0.8472 ± 0.0229</u> | 0.8414 ± 0.0125 | 0.8405 ± 0.0206 | 0.8574 ± 0.0247 | |
| 100% | 0.8419 ± 0.0129 | 0.8416 ± 0.0200 | <u>0.8621 ± 0.0257</u> | 0.8442 ± 0.0185 | 0.8605 ± 0.0303 | 0.8521 ± 0.0343 | 0.8518 ± 0.0205 | 0.8725 ± 0.0294 | |
| Healthcare | 25% | <u>0.7410 ± 0.0101</u> | 0.7403 ± 0.0070 | 0.7402 ± 0.0078 | 0.7385 ± 0.0073 | 0.7397 ± 0.0048 | 0.7285 ± 0.0068 | 0.7311 ± 0.0071 | 0.7469 ± 0.0052 |
| 50% | <u>0.7514 ± 0.0065</u> | 0.7478 ± 0.0068 | 0.7489 ± 0.0067 | 0.7460 ± 0.0075 | 0.7487 ± 0.0064 | 0.7387 ± 0.0068 | 0.7393 ± 0.0080 | 0.7548 ± 0.0059 | |
| 75% | <u>0.7539 ± 0.0061</u> | 0.7519 ± 0.0056 | 0.7516 ± 0.0057 | 0.7500 ± 0.0053 | 0.7523 ± 0.0051 | 0.7441 ± 0.0051 | 0.7409 ± 0.0072 | 0.7573 ± 0.0045 | |
| 100% | <u>0.7554 ± 0.0058</u> | 0.7526 ± 0.0049 | 0.7529 ± 0.0063 | 0.7514 ± 0.0040 | 0.7537 ± 0.0044 | 0.7471 ± 0.0052 | 0.7422 ± 0.0069 | 0.7585 ± 0.0050 | |
| Teen | 25% | 0.8237 ± 0.0482 | 0.8369 ± 0.0404 | 0.8267 ± 0.0409 | 0.8453 ± 0.0491 | 0.7036 ± 0.0603 | 0.8474 ± 0.0336 | <u>0.8521 ± 0.0563</u> | 0.8554 ± 0.0360 |
| 50% | 0.9025 ± 0.0469 | <u>0.9213 ± 0.0467</u> | 0.8876 ± 0.0373 | 0.9403 ± 0.0392 | 0.7223 ± 0.0477 | 0.8918 ± 0.0475 | 0.9054 ± 0.0286 | 0.9151 ± 0.0400 | |
| 75% | 0.9143 ± 0.0377 | 0.9318 ± 0.0385 | 0.9350 ± 0.0212 | <u>0.9373 ± 0.0249</u> | 0.7867 ± 0.0641 | 0.9251 ± 0.0300 | 0.9286 ± 0.0326 | 0.9511 ± 0.0268 | |
| 100% | 0.9398 ± 0.0203 | 0.9514 ± 0.0267 | 0.9285 ± 0.0281 | <u>0.9589 ± 0.0180</u> | 0.8071 ± 0.0444 | 0.9413 ± 0.0217 | 0.9458 ± 0.0361 | 0.9610 ± 0.0303 | |
| Urban | 25% | <u>0.5926 ± 0.0065</u> | 0.5832 ± 0.0105 | 0.5856 ± 0.0082 | 0.5812 ± 0.0105 | 0.5790 ± 0.0078 | 0.5794 ± 0.0094 | 0.5777 ± 0.0092 | 0.5964 ± 0.0045 |
| 50% | <u>0.6069 ± 0.0037</u> | 0.6030 ± 0.0053 | 0.5988 ± 0.0059 | 0.6011 ± 0.0075 | 0.6008 ± 0.0084 | 0.6021 ± 0.0065 | 0.5912 ± 0.0045 | 0.6113 ± 0.0054 | |
| 75% | <u>0.6201 ± 0.0031</u> | 0.6089 ± 0.0051 | 0.6091 ± 0.0040 | 0.6081 ± 0.0041 | 0.6072 ± 0.0025 | 0.6094 ± 0.0052 | 0.6029 ± 0.0056 | 0.6250 ± 0.0034 | |
| 100% | <u>0.6240 ± 0.0036</u> | 0.6131 ± 0.0055 | 0.6112 ± 0.0049 | 0.6151 ± 0.0066 | 0.6144 ± 0.0066 | 0.6167 ± 0.0040 | 0.6110 ± 0.0070 | 0.6277 ± 0.0041 |
W2 Value Protection, Fallback Sampling, and Combination Validity
Following your suggestion for direct evaluation, we conducted two types of checks on the final generated samples; below, we describe violations of domain rules and changes in statistical structure, respectively.
(1) Based on the data content and existing domain knowledge, we identified domain rules that could be explicitly checked and checked the final samples generated by the complete pipeline. Because these public datasets do not provide accurate domain-rule tables, we have not yet identified rules suitable for accurate evaluation for some datasets, and we mark these cases as N/A in the table. DIDA reports zero violation rates for all assessed rules and is the only augmentation method in the table to achieve this across all of them, indicating that the final generated samples perform relatively well under these rules.
Table: Results of checks against verifiable domain rules
| Dataset | Rule | Original | SMOTE | CTGAN | SMOTE-CDNN | TabDDPM | MDLM-local | TANDEM | DIDA |
|---|---|---|---|---|---|---|---|---|---|
| Alzheimers | BP order | 8.3786 ± 0.3801 | 6.3510 ± 0.9000 | 11.6725 ± 9.0009 | 5.7808 ± 0.7666 | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | 9.4710 ± 4.5862 | 0.0000 ± 0.0000 |
| Kidney | BP order | 7.7961 ± 0.7695 | 5.6445 ± 1.0581 | 7.8612 ± 4.7513 | 5.9319 ± 1.0217 | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | 5.3982 ± 2.6296 | 0.0000 ± 0.0000 |
| Heart | Not assessed | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| Healthcare | BMI identity | 0.0000 ± 0.0000 | 1.9397 ± 0.2569 | 98.0215 ± 0.3980 | 2.0820 ± 0.2150 | 90.2260 ± 5.0925 | 7.7325 ± 0.9199 | 96.0295 ± 0.5929 | 0.0000 ± 0.0000 |
| BP order | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | 0.0023 ± 0.0061 | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | 0.1241 ± 0.2278 | 0.0000 ± 0.0000 | |
| Teen | Not assessed | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| Urban | Not assessed | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
Thank you for carefully reviewing our paper and recognizing the practical significance of small-sample settings and settings with strong constraints, as well as the interpretability and auditability of value-aware augmentation. Your questions about statistical boundaries, hard business rules, and the validity of combinations after fallback have also prompted us to further consider the relationships among these design choices; below, we explain them in terms of our design considerations and supplementary results.
W1 Dynamic Domain Correction, Hard Business Rules, and Cross-Feature Constraints
In response to your comments on domain correction and business constraints, we discuss three aspects: the considerations behind introducing this mechanism, issues that currently require further study, and experiments in small-sample settings.
(1) We introduced domain correction mainly because statistical boundaries in small-sample settings may be too narrow. The range obtained using the interquartile range reflects the region where the current data are concentrated, and some genuine, rare observations at the margins that carry important predictive information may fall outside this range. If these observations are directly excluded, subsequent augmentation can only take place within a further narrowed range, and the already limited information at the margins can easily be lost.
(2) Regarding your suggestion to distinguish statistical outliers from violations of business rules and to validate constraints after correction, we believe that more detailed mathematical modeling is still needed to clarify the relationships among statistical boundaries, sample value, and business constraints. The difficulty is that adjusting the range of a single feature may also affect the validity of its combinations with other features, while the lack of complete domain-rule descriptions in public datasets makes both modeling and validation more difficult. In future work, we will continue to study these relationships and improve joint constraint checks after domain correction and sample generation, to enhance the safety and applicability of the method in settings with strong constraints.
(3) At this stage, we use supplementary experiments at different training fractions to evaluate DIDA's predictive performance in small-sample settings more fully, with the results shown in the table below. DIDA achieves the highest mean ROC-AUC for most combinations of training size and dataset, and also maintains its lead at the smallest training fraction on all six datasets. This shows that the complete method still yields predictive benefits when training information is limited, supporting the practical significance of our design of DIDA for small-sample settings.