acceptodds
Under review as a conference paper at ICLR 2027

Learning When Structure-Based Semantics Help Tabular Imputation

Abstract

Pretrained language models provide rich semantic priors for tabular imputation, but their utility is inherently conditional: general semantic associations may conflict with dataset-specific dependencies or become redundant when the observed values already provide sufficient evidence. Existing language-enhanced imputers therefore face a fundamental mismatch between general-purpose semantic priors and the instance-specific evidence required for each missing entry. To address this mismatch, we propose SemImputer, a structure-based semantic representation imputation framework that learns when semantic priors provide complementary evidence for each missing-value prediction. Specifically, SemImputer first aligns semantic representations with tabular structures through structure-semantic representation alignment, then encodes the missingness context for each target through context-aware missing pattern encoder, and finally learns whether semantic information improves each prediction through counterfactual utility-guided fusion for selective semantic injection. Across 10 real-world datasets, SemImputer achieves state-of-the-art performance, reducing RMSE by 5.4% and improving R² by 16.1% on average over the strongest baseline, while remaining robust across heterogeneous missingness mechanisms without mechanism-specific retraining.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Tabular Imputation, Pretrained Language Models, Semantic-Structural Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TYgGOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses the observation that semantic priors from pretrained language models are not always beneficial for tabular imputation and proposes SemImputer to incorporate such information selectively. SemImputer consists of Structure-Semantic Alignment (SSA), a Missing Pattern Encoder, and Counterfactual Utility-Guided Fusion (CUF). In particular, CUF trains a utility gate using the difference in prediction errors with and without semantic injection. Across 10 datasets under MCAR, MAR, MNAR, and MIXED missingness, the paper reports average improvements of 5.4% in RMSE and 16.1% in R2R^2 over existing methods.

Critical strengths
  • Meaningful problem formulation: The paper recognizes that semantic priors are not universally useful and proposes selectively incorporating semantic information depending on the prediction target.
  • Strong empirical performance: The paper reports consistent average improvements over existing methods on 10 datasets. Appendix C.1 also reports statistically significant dataset-level improvements over all baselines for RMSE and R2R^2.
  • Extensive additional experiments: The paper provides component ablations and evaluations under unseen missing rates and held-out missingness mechanisms, as well as comparisons against fixed-view
Critical weaknesses
  • Insufficient justification for utility-guided fusion: Equation (4) constructs binary utility labels, whereas Equation (6) uses the predicted probability of these labels as a continuous fusion weight. The paper lacks theoretical or empirical evidence that the probability of semantic benefit determines an appropriate injection strength.
  • Unclear contribution of counterfactual supervision: In Table 3, AUROC improves substantially from 0.543 to 0.910, while RMSE improves by only approximately 1.3%. Moreover, the ablation also changes the gradient flow and training configuration, making it difficult to isolate the independent contribution of counterfactual supervision.
  • Limited validation on genuinely incomplete data: All main experiments apply artificial missingness to complete datasets. It remains unverified whether the proposed utility supervision is effective when ground-truth values at naturally missing training entries are unavailable.
Questions for authors
  • Q1. Utility probability and injection strength: What justifies using the predicted probability of a binary counterfactual utility label as a continuous fusion weight? Are there comparisons with a fixed injection strength or a continuous utility target?
  • Q2. Independent effect of counterfactual supervision: The ablation in Table 3 changes not only the utility loss but also the gradient flow. Can the authors provide a comparison that controls for other training conditions? Also, what exactly distinguishes “w/o CUF” in Figure 4 from “w/o Counterfactual Supervision” in Table 3?
  • Q3. Validity of field-level alignment: Equation (1) applies alignment loss to both observed and masked fields. Is it appropriate to align identical semantic descriptors with row-specific structural representations? Are comparisons with observed-only or missing-only alignment available?
  • Q4. Applicability to genuinely incomplete data: How are utility labels constructed when the training data itself contains missing values? Does the improvement persist when the model is trained by additionally masking observed entries?
4 comments
AuthorsOpenReview ↗

Response to Reviewer TYgG (Part 1)

Thank you for your constructive suggestions. The following experiments use test rates of 30/40/50% under MCAR/MAR/MNAR/MIXED on the paper’s ten datasets. Native tests additionally mask observed entries and retain original missingness. Fixed-strength controls use seed-42 checkpoints. Full-model retraining uses seeds 42/84/126. Means average available settings and seeds within each dataset; macro means weight datasets equally.

W1 & Q1

CUF trains a sigmoid gate with BCE to predict whether capped injection improves a target's loss over zero injection beyond the prescribed margins. The predicted probability of semantic benefit scales a type-specific cap learned through reconstruction, giving a continuous interpolation between zero and capped injection. Identifying the loss-minimizing intermediate strength additionally requires losses at intermediate strengths.

Our fixed-strength controls change only the injection coefficient on the frozen checkpoints: zero, half-cap, full-cap or the original CUF gate. All other parameters and inputs remain fixed.

Injection policyRMSE ↓MAE ↓R2R^2 ↑WD ↓Acc (%) ↑
Zero injection0.780360.432050.407410.2367870.83
Half injection0.760280.422380.417220.2360471.31
Capped injection0.760550.422990.426770.2371171.61
CUF0.754440.419740.420830.2207472.72

Separately, we freeze a common reference predictor and its caps and train only new gates and context projections. Binary utility uses BCE; strength regression uses MSE toward the lowest-loss fraction among 0, 0.05, 0.1, 0.25, 0.5, 0.75 and 1 for each known training target, selected using numerical Huber loss or categorical cross-entropy. Initialization, masks, optimizer, budget and validation selection are matched.

Gate settingRMSE ↓MAE ↓R2R^2 ↑WD ↓Acc (%) ↑
Binary utility0.754440.419740.420830.2207472.72
Injection strength regression0.750160.421700.427310.2256172.04

CUF leads fixed policies on RMSE, MAE, WD and Acc; capped injection gives the highest R². Strength regression improves RMSE and R², while binary utility gives better MAE, WD and Acc. We will combine endpoint-benefit supervision with intermediate-strength loss differences to better capture beneficial partial injection.

W2 & Q2

CUF's core contribution is explicit counterfactual supervision of target-specific semantic benefit. Section 3.3 states that CUF “explicitly supervises this predictive gain, which reconstruction alone leaves implicit.”

Figure 4 removes the entire CUF module before training and independently trains the remaining model. Table 3 retains the gate and semantic fusion but removes utility supervision, allowing reconstruction to train the gate. Gate-weighted row pooling also supplies row-alignment gradients. Initialization and checkpoint settings are documented in Appendix D.1. This compares explicit utility learning with end-to-end gating.

Our independent reruns match initial ρ=0.5, unweighted row pooling, detached aligned gate inputs, data, masks, architecture, optimizer, curriculum and checkpoint rules within each study. Full uses BCE and stops reconstruction gradients at the gate. Recon removes BCE and enables those gradients. Fixed half removes BCE and keeps them stopped, fixing ρ=0.5 while other parameters and caps train. Full–Fixed half tests utility supervision under the same reconstruction-gradient rule.

SettingGateRMSE ↓MAE ↓R2R^2 ↑WD ↓Acc (%) ↑
10 datasetsFull0.750110.412280.427460.2151172.86
10 datasetsFixed half0.750920.412520.426280.2172872.84
10 datasetsRecon0.750440.413240.427100.2163372.84
Cervical CancerFull0.96640.50250.24060.380395.59
Cervical CancerFixed half0.97030.50380.23280.387995.68
Cervical CancerRecon0.97020.50160.23340.384895.70
Mammographic MassFull0.92710.75840.12720.477263.17
Mammographic MassFixed half0.93170.75620.11950.478863.22
Mammographic MassRecon0.93080.75540.12090.478363.35

Full leads all five macro means on the ten datasets, with RMSE reductions of 0.04% over Recon and 0.11% over Fixed half. On both native datasets, Full improves mean RMSE, R² and WD over both controls; Recon gives lower MAE and higher categorical Acc.

AuthorsOpenReview ↗

Response to Reviewer TYgG (Part 2)

Thank you for your careful assessment. The alignment and native-missingness studies use seeds 42/84/126 and test rates of 30/40/50%; in SemImputer’s native training, 10/20/30/40% of observed entries are masked. Results are averaged over settings and seeds, with datasets weighted equally. Native-missingness results are reported as mean ± standard deviation across the three seeds.

Q3

Observed-cell texts include values; masked descriptors express field meaning shared across rows. Structural tokens encode both field identity and row context, so the descriptor serves as a shared semantic prior. Eq. (1) aligns projected tokens through an auxiliary cosine loss with a decaying weight, while reconstruction and row-level InfoNCE provide row-specific supervision. We compare all-field, observed-only, masked-only and no alignment on four datasets under MCAR/MAR/MNAR/MIXED, normalizing by selected fields and keeping other settings fixed.

Field-alignment scopeRMSE ↓MAE ↓R² ↑WD ↓Acc (%) ↑
All fields0.72310.36640.46410.195380.05
Masked only0.74280.38930.46050.203379.09
Observed only0.72930.37490.45270.204779.11
No field alignment0.73910.38510.45910.218278.25

Joint alignment leads all five means: RMSE falls 2.16% and Acc rises 1.80 percentage points versus no field alignment. Adding masked-field alignment to observed-only alignment lowers RMSE from 0.7293 to 0.7231, while masked-only RMSE is higher than without field alignment. These results support shared-descriptor alignment within the joint observed/masked objective.

W3 & Q4

Our benchmark follows CACTI and UnIMP (Section 4.1). We evaluated all rows of Cervical Cancer [1] (858 rows, 32 inputs, 13.19% missing) and Mammographic Mass [2] (961 rows, four inputs, 4.16% missing), excluding outcome/screening labels.

Following your suggestion, we retain native missingness and additionally mask observed values. Zero/capped predictions share inputs and parameters; their loss differences on known masked targets define Eq. (4)'s labels. Only these targets supervise reconstruction and utility. Methods share stratified splits and test masks, with training-only preprocessing. Metrics score known masked entries. CtrTab is omitted: its evaluated formulation needs complete targets.

Cervical Cancer

MethodRMSE ↓MAE ↓R² ↑WD ↓Cat. Acc (%) ↑
ReMasker0.9456 ± 0.31550.4954 ± 0.07990.2803 ± 0.12250.3587 ± 0.065395.88 ± 0.91
KPI1.1077 ± 0.27030.5514 ± 0.07930.0448 ± 0.02640.3481 ± 0.065795.66 ± 0.41
DiffPuter0.9671 ± 0.30400.4657 ± 0.07820.2612 ± 0.10670.2982 ± 0.064496.51 ± 0.16
UnIMP1.0283 ± 0.30970.5349 ± 0.09110.1619 ± 0.09970.4170 ± 0.104895.30 ± 0.55
CACTI1.0232 ± 0.50960.5777 ± 0.28410.2712 ± 0.23470.4634 ± 0.296090.73 ± 12.75
RefiDiff1.0890 ± 0.27010.6726 ± 0.05690.1009 ± 0.04430.4590 ± 0.068894.79 ± 0.70
SemImputer0.9664 ± 0.27930.5025 ± 0.06310.2406 ± 0.06620.3803 ± 0.033395.59 ± 0.22

Mammographic Mass

MethodRMSE ↓MAE ↓R² ↑WD ↓Cat. Acc (%) ↑
ReMasker0.9407 ± 0.09870.7682 ± 0.11400.1170 ± 0.03970.4908 ± 0.053163.90 ± 2.04
KPI0.9510 ± 0.12910.7778 ± 0.13740.1073 ± 0.06250.4639 ± 0.060260.83 ± 1.27
DiffPuter1.0307 ± 0.08890.8378 ± 0.09980.0658 ± 0.05130.4611 ± 0.112860.91 ± 2.37
UnIMP0.9252 ± 0.10560.7448 ± 0.11760.1352 ± 0.05170.5015 ± 0.164862.31 ± 5.61
CACTI0.9173 ± 0.09830.7445 ± 0.10270.1408 ± 0.05480.5251 ± 0.097364.82 ± 4.46
RefiDiff0.9682 ± 0.11660.7747 ± 0.11930.1037 ± 0.05450.3566 ± 0.070361.49 ± 3.12
SemImputer0.9271 ± 0.09790.7584 ± 0.11640.1272 ± 0.03570.4772 ± 0.065763.17 ± 2.78

SemImputer's mean Cervical RMSE ranks second, 5.55% below CACTI and 6.02% below UnIMP. On Mammographic Mass, RMSE is 1.45% below ReMasker's and WD is below both semantic baselines. With cached semantics, inference needs one gated prediction; Fig. 6 on Default reports the second-lowest online latency, 0.02 GFLOPs/row and peak GPU memory below 100 MB.

[1] Kelwin Fernandes, Jaime Cardoso, and Jessica Fernandes. Cervical Cancer (Risk Factors). UCI Machine Learning Repository, 2017.

[2] Matthias Elter. Mammographic Mass. UCI Machine Learning Repository, 2007.

The authors’ responses and additional experiments have partially addressed the earlier questions. However, concerns about the novelty of the proposed model and the stability of the experimental results remain.

Additionally, the procedure for selecting the weights of the six loss terms in Equation (7) is unclear. Although the schedule for changing the weights during training and their final values are provided, it is difficult to determine from the main text, appendix, or provided code what criteria and search procedure were used to select those final values. Please explain the search range, selection criteria, and tuning budget for each weight. Please also clarify how much performance varies with the choice of the main weights and whether a comparable tuning budget was applied to the baseline methods.

AuthorsOpenReview ↗

Thank you very much for your feedback!

Thank you for your follow-up comments. We address your remaining concerns below.

Q1 What is the core novelty of the model?

Existing semantic imputers leave a key question implicit: does semantics help predict each missing target? Our core novelty is to explicitly learn whether semantic information improves the prediction of each missing target given the available observations. Our method constructs counterfactual supervision by comparing the prediction losses of the same model with and without semantic injection. The predicted probability of benefit then regulates injection to improve imputation while limiting interference from unhelpful semantics.

Q2 How stable are the experimental results?

Our ablation and rebuttal comparisons include three random seeds (42/84/126). Appendix C.1 additionally reports dataset-level paired comparisons: RMSE and R² improvements over all seven baselines remain significant after Holm correction. Appendix C.2 evaluates 60% and 70% missingness, beyond the training range, and Appendix C.3 evaluates missingness mechanisms unseen during training. Overall, we consider the experimental results of our method to be reasonably stable.

Q3 How were the six loss weights selected?

Appendix B.7 specifies the resulting schedule and coefficient values. The initial weights were set empirically. Checkpoints are selected by minimizing the unweighted sum of numerical and categorical validation reconstruction losses.

Q4 How sensitive is performance to the main loss weights?

Figure 5 varies the final utility-loss weight after warm-up (0.1, 0.2 or 0.4). Each setting keeps the weight at zero during alignment, increases it linearly over the first 10 transition epochs, and holds it constant thereafter.

Q5 Were comparable tuning budgets used for the baselines?

Our evaluation follows the benchmark datasets used by CACTI and UnIMP. We adopt the baselines’ official default hyperparameters, with benchmark adaptations and training budgets documented in Appendix B.8. No additional hyperparameter search was performed for the baselines.

Reviewer_7HQwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies when language-derived semantic information improves tabular imputation beyond the dependencies learned from observed values. SemImputer combines frozen language embeddings with a Transformer-based structural encoder and aligns their representations at the field and row levels. Its main component, Counterfactual Utility-Guided Fusion (CUF), compares two predictions from the same model, with the semantic residual either disabled or enabled at a learned upper limit. The difference in prediction loss yields a binary utility label that supervises a gate controlling the strength of semantic injection. The model also incorporates missingness context and is trained across multiple missingness mechanisms and rates.

Experiments cover ten datasets and seven baselines under four missingness mechanisms and three test rates. The reported average RMSE improvement over the strongest tested baseline in each condition is approximately 5.44%. Ablations and utility diagnostics assess the proposed components.

Critical strengths

(1) The paper highlights the additional value of semantics as a distinct problem in tabular imputation.

The paper's useful insight is that semantic injection cannot always help the inputation. Simply aligning semantic and tabular representations is not enough. This motivates learning semantic utility for each target and missingness context, extending merely focus on the accuracy of fused predictions. Compared to recent LLM-based semantics injection tabular imputation method, the paper really produces new perspective.

(2) The CUF component introduces explicit utility supervision for the semantic gate.

It compares prediction errors with the semantic residual disabled and enabled. The resulting labels tell the gate whether semantic injection improves a masked target's prediction. This ties gate training directly to the measured benefit of the semantic residual, giving a concrete method for controlling its contribution to each missing entry. And it requires only one gated prediction at inference.

Critical weaknesses

(1) The benefit of target-specific semantic injection remains unclear.

CUF may improve imputation by identifying targets that benefit from semantics, or by reducing semantic injection overall. Figure 3 supports the gate's prediction of endpoint utility, but does not distinguish these explanations for the final imputation gains. (1.1) Using the same trained model, could the authors compare the learned gates with a constant gate for each field and with gate scores shuffled across masked targets within each field? Preserving each field's mean injection strength would help isolate the value of target-specific allocation. (1.2) Furthermore, the paper also does not report aggregate imputation results for the two endpoint controls: full lower-bound injectionand full upper-bound injection. A direct comparison among these two controls and the learned target-specific gate would clarify whether the gain comes from adding semantics at all, from reducing injection below the upper bound, or from assigning different injection strengths across targets.

(2) CUF's contribution relative to existing fusion methods needs clearer evidence.

As prior work has explored gating-based fusion method [1,2], and counterfactual contribution-based gating [3]. Adapting their fusion principles may be more informative in tabular imputation. In conclusion, the remaining question is what CUF's endpoint-derived utility labels add over recent representative alternative fusion objectives. A clearer explanation of the methodological differences, together with a targeted comparison under the same backbone, would help establish this contribution.

(3) The ablation does not separate utility supervision from changes to row alignment.

Both variants in Table 3 retain a gate for semantic injection. Full CUF trains the gate with utility labels and uses unweighted mean pooling for the auxiliary semantic row representation; the control uses gate-weighted pooling. The control gate therefore learns from reconstruction and, during early training, row-alignment loss. Appendix D.1 discloses this change but does not explain why the control needs a different pooling rule. The RMSE reduction from 0.7296 to 0.7201 supports the complete training configuration, while leaving the contribution of utility supervision partly unresolved. Keeping the same unweighted row pooling in both variants, with the control gate trained through reconstruction, would make the attribution clearer.

(4) Discussion: practical significance of the RMSE gains.

The evaluation focuses on imputation accuracy. As a minor discussion point, could the authors comment on how the reported RMSE gains might translate into downstream classification or regression performance? A brief discussion of the potential benefits and limitations would help put these gains in context.

[1] Qingyang Zhang et al. Provable Dynamic Fusion for Low-Quality Multimodal Data. ICML, 2023.

[2] Sijie Li, Chen Chen, and Jungong Han. SimMLM: A Simple Framework for Multi-modal Learning with Missing Modality. ICCV, 2025.

[3] Yunping Shi, En Yu, Kairui Guo, and Jie Lu. GAUGE: Granularity-Adaptive Counterfactual Gating of Evidence for Incomplete Multimodal Classification. arXiv:2608.05608, 2026. Preprint.

Questions for authors

Please see the Critical Weaknesses section above for my questions and suggested comparisons.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 7HQw (Part 1)

Thank you for your constructive suggestions on target-specific semantic injection and fusion objectives. The following two studies cover the paper’s ten datasets and 12 missingness scenarios.

W1

Figure 3 shows that capped injection benefits 38% of target evaluations and that gate probabilities are calibrated to the stricter utility-label event (ECE = 0.004). These observations motivate selective injection; the following controls test how injection strength and target-specific allocation contribute to final imputation performance.

We evaluate the same frozen seed-42 checkpoints. We replace masked targets' gate scores with their field mean or shuffle them within each field, preserving each field's mean injection strength; shuffling also preserves its score distribution (20 repetitions). We also compare zero injection (the lower bound), injection at the learned type-specific cap (the upper bound), half-cap injection and CUF. All model parameters and inputs remain fixed, and results are averaged across scenarios and datasets.

Evaluation controlRMSE ↓MAE ↓R2R^2 ↑WD ↓Acc (%) ↑
Field-mean gate0.770070.422290.427470.2253071.86
Within-field shuffled gate0.780760.432290.417460.2353070.85
Zero injection0.780360.432050.407410.2367870.83
Half injection0.760280.422380.417220.2360471.31
Capped injection0.760550.422990.426770.2371171.61
CUF (learned gate)0.754440.419740.420830.2207472.72

Capped injection reduces RMSE from 0.78036 to 0.76055 relative to zero injection, while CUF improves all five metrics over half injection. With mean injection strength preserved, CUF outperforms both allocation controls on RMSE, MAE, WD and Acc, and shuffling worsens all five metrics. The field-mean gate gives the highest R². These results support the value of assigning semantic injection to specific targets and contexts.

W2

QMF estimates modality quality from uncertainty, SimMLM's More-vs-Fewer (MoFe) objective encourages lower loss with more modalities, and GAUGE estimates evidence-replacement effects through Taylor approximations. CUF directly compares zero and capped semantic injection for each masked training target, turning loss reductions that exceed the prescribed margins into explicit utility labels. BCE trains the gate to predict this benefit from the target representations and observed context, and the predicted probability of semantic benefit scales a learnable type-specific injection cap.

We adapted SimMLM's MoFe objective to branches with and without semantic injection, training both through reconstruction and penalizing cases where adding semantics increases the loss. All three objectives share the backbone, semantic inputs, unweighted pooling, initialization, curriculum and selection rule. This comparison uses three seeds, averaging across scenarios, seeds and datasets.

Gate objectiveRMSE ↓MAE ↓R2R^2 ↑WD ↓Acc (%) ↑
Reconstruction0.760440.433240.417100.2363371.84
MoFe adaptation0.759440.435820.42790.2356771.97
CUF0.754440.419740.420830.2207472.72

CUF achieves the lowest mean RMSE, MAE and WD and highest categorical accuracy, while the MoFe adaptation gives the highest R². Relative to MoFe, CUF reduces RMSE by 0.66% and improves accuracy by 0.75 percentage points. This comparison supports the contribution of explicit target-specific utility supervision beyond reconstruction and the adapted MoFe ranking objective.

AuthorsOpenReview ↗

Response to Reviewer 7HQw (Part 2)

Thank you for your suggestions on isolating utility supervision and clarifying the practical significance of imputation gains.

W3

The original ablation compared a gate trained jointly through reconstruction and row alignment with a gate trained by explicit utility supervision. Gate-weighted pooling allowed the control's gate to receive row-alignment gradients, whereas CUF used unweighted pooling to keep those gradients from directly training its gate. Reconstruction-based gating can also use unweighted pooling.

Following your suggestion, we use unweighted row pooling in all three variants and fix gate initialization and checkpoint eligibility. The controls remove utility BCE and either train the gate through reconstruction or fix it at ρ=0.5\rho=0.5, while the remaining model and type caps still train. We evaluated these variants over 10 datasets and 12 missingness scenarios, averaging across scenarios.

Gate trainingRMSE ↓MAE ↓R2R^2 ↑WD ↓Acc (%) ↑
Reconstruction, unweighted pooling0.775440.433240.407100.2363371.84
Fixed half, unweighted pooling0.764920.422520.416280.2272871.84
CUF, unweighted pooling0.754440.419740.420830.2207472.72

CUF achieves the best average performance across all five metrics, reducing RMSE by 2.71% and 1.37% relative to the reconstruction-trained and fixed gates, respectively, and improving categorical accuracy by 0.88 percentage points over both. These results support the contribution of explicit utility supervision with row pooling held fixed.

W4

We illustrate the downstream implications using Students three-class classification and Bike rental-count regression from our original benchmark. We retrain imputers with outcome labels excluded and use fixed random forest classifiers/regressors on their completed features. Imputation and downstream metrics use the same completed test tables. Results are mean±SD over seeds 42/84/126, averaging the 12 MCAR/MAR/MNAR/MIXED settings at 30/40/50% missingness within each seed.

Students: three-class classification

MethodImp. RMSE ↓Acc (%) ↑
ReMasker0.6603±0.025873.92±0.55
UnIMP0.7211±0.030673.42±0.44
SemImputer0.5805±0.019374.04±0.56

Bike: rental-count regression

MethodImp. RMSE ↓Pred. RMSE (count) ↓Pred. RMSE (std.) ↓
ReMasker0.7022±0.0291363.57±3.190.5621±0.0078
UnIMP0.7416±0.0242374.02±6.420.5782±0.0126
SemImputer0.7176±0.0196370.80±4.340.5732±0.0093

Imputation RMSE measures masked numerical errors on standardized features; Bike prediction RMSE is reported in rental-count units and standardized by the training-target standard deviation. Lower reconstruction error can improve the inputs used for classification or regression. On Students, a 12.1% lower imputation RMSE than ReMasker accompanies a small Acc increase from 73.92% to 74.04%: feature corrections can improve reconstruction while leaving predicted classes unchanged. For regression, the benefit is expressed in the target quantity: SemImputer reduces Bike prediction RMSE from UnIMP's 374.02 to 370.80, while ReMasker remains stronger at 363.57. These results illustrate modest, task-dependent downstream gains. Their magnitude depends on which features improve, their relevance to the outcome and the categorical information used by the predictor, so practical value is best assessed through task-level accuracy or prediction error alongside imputation RMSE.

Reviewer_wMoVOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SemImputer for tabular data imputation. It aligns frozen language embeddings with tabular representations and encodes the current missingness context. A utility gate controls the semantic residual for each missing entry. The gate learns from paired predictions with zero and capped semantic injection. Training interleaves several missingness mechanisms and rates. Experiments on ten datasets report better average imputation accuracy than seven baselines, together with utility diagnostics and tests on unseen missingness settings.

Critical strengths

S1: The paper asks a useful question: when semantic information helps a particular imputation. Conditioning the gate on observed evidence and missingness context makes the approach relevant to varying observation patterns.

S2: Comparing zero and capped semantic injection provides an explicit utility signal. The accompanying gate diagnostics help examine what the gate learns beyond the final imputation scores.

S3: The empirical comparison shows consistent average RMSE gains across the main settings. Shared training views for semantic baselines and tests on unseen missingness mechanisms strengthen the evaluation.

Critical weaknesses

W1: Endpoint supervision may discourage useful partial injection. Eq. (4) compares zero and capped injection, while Eq. (6) uses an intermediate scale. A negative endpoint label can suppress a smaller beneficial residual. The reconstruction loss trains the residual and cap, but its gradients to the gate are stopped.

W2: The ablation in Table 3 does not fully isolate utility supervision. Appendix D.1 shows that removing it also changes gate initialization, semantic row pooling, and checkpoint eligibility. Holding these settings fixed would make its contribution clearer.

W3: The AUROC/AUPRC comparison in Table 3 uses checkpoint-specific utility labels (Appendix B.9). Higher scores therefore do not isolate better utility estimation: both the scoring model and the event being predicted change. A comparison with fixed reference predictors would make this claim more direct.

Questions for authors

Q1: How does the gate handle targets for which capped injection is harmful but partial injection helps? For a fixed scalar predictor P(t)=tP(t)=t with t∈[0,1]t\in[0,1] and target 0.40.4, the endpoint label is negative although t=0.4t=0.4 minimizes reconstruction loss. Can supervision over intermediate scales address this case?

Q2: What is the effect of removing utility supervision when gate initialization, row pooling, and checkpoint eligibility are held fixed?

Q3: Can utility estimators be compared using the same fixed pair of zero-injection and capped-injection predictors, so that evaluation labels are shared?

1 comment
AuthorsOpenReview ↗

Response to Reviewer wMoV

Thank you for your constructive suggestions. The following experiments cover 10 datasets and 12 missingness scenarios, with results averaged across scenarios.

W1 & Q1

CUF trains a sigmoid gate with BCE to predict whether capped injection improves upon zero injection beyond the specified margins. Its continuous benefit probability scales a type-specific cap learned through reconstruction. Supervision at intermediate injection strengths offers finer targets, but small loss differences between similarly effective strengths can produce widely different regression labels. Our control freezes the predictor and caps and trains only the gate and its context projection with MSE toward the lowest-loss cap fraction among 0, 0.05, 0.1, 0.25, 0.5, 0.75 and 1, using Huber loss for numerical targets and cross-entropy for categorical targets to select the fraction.

Gate settingRMSE ↓MAE ↓R2R^2 ↑WD ↓Acc (%) ↑
Binary utility0.754440.419740.420830.2207472.72
Injection strength regression0.750160.421700.427310.2256172.04

Regression improves RMSE and R², while binary utility yields lower MAE and WD and higher categorical accuracy. Neither objective dominates across all metrics.

W2 & Q2

We fix gate initialization, unweighted row pooling and checkpoint eligibility, and compare CUF with two controls without utility supervision: a gate trained through reconstruction and a fixed gate (ρ=0.5\rho=0.5). The remaining model and type caps are trained in all three variants.

Gate trainingRMSE ↓MAE ↓R^2 ↑WD ↓Acc (%) ↑
Reconstruction0.775440.433240.407100.2363371.84
Fixed half0.764920.422520.416280.2272871.84
CUF0.754440.419740.420830.2207472.72

CUF leads all five averaged metrics, reducing RMSE by 2.71% and 1.37% relative to Reconstruction and Fixed half, respectively, and improving accuracy by 0.88 percentage points over both. This supports the benefit of utility supervision under matched settings.

W3 & Q3

We separately use Full CUF and a reconstruction-trained model as frozen references. Within each reference, both estimators share fixed zero/capped predictions and identical utility labels. We train only new gates and their context projections, using BCE or reconstruction loss, with matched initialization, masks and training settings. Reconstruction uses Huber loss for numerical targets and cross-entropy for categorical targets. This study also averages over three seeds (42, 84 and 126).

Fixed referenceGate trainingNum. AUROC ↑Num. AUPRC ↑Cat. AUROC ↑Cat. AUPRC ↑
Full CUFBinary utility0.82980.14780.82590.3235
Full CUFReconstruction0.53810.04070.55160.1254
Reconstruction-trainedBinary utility0.79640.11710.78940.4645
Reconstruction-trainedReconstruction0.53820.04470.51080.2100

Under both fixed references, binary utility achieves higher mean AUROC and AUPRC for both target types. These shared-label comparisons support better recognition of the same counterfactual semantic benefit.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for their professional review and valuable opinions and suggestions. Regarding the questions you raised, we have provided detailed answers one by one in our response for your review.

Furthermore, we have uploaded the revised paper, and here, we summarize the updated content. All changes in the revised PDF are marked in blue.

Reviewer TYgG

  • Section 3.3 clarifies why the probability of benefit at the cap guides injection but cannot determine the best intermediate strength.
  • Section 4.4 distinguishes module removal from supervision ablation and adds matched three-seed controls. Section 4.2 summarizes results on two naturally incomplete datasets, with full comparisons in Appendix C.
  • Appendix B.7 states that the initial loss weights were set empirically. Section 3.1 explains why masked fields share descriptors, and Appendix D compares four alignment settings.
  • Appendix B.8 clarifies that we used the baselines’ default hyperparameters without an additional search.

Reviewer 7HQw

  • A new table in Section 4.3 compares learned, field-mean, and within-field shuffled gates to test target-specific allocation. Appendix D reports fixed-strength controls.
  • Section 4.4 compares CUF with reconstruction-trained gating on the same backbone. Our reviewer response also compares CUF with prior fusion objectives.
  • A new table in Section 4.4 compares utility-supervised, reconstruction-trained, and fixed gates with matched row pooling, initialization, and checkpoint selection.
  • Section 4.2 clarifies why lower imputation RMSE can have a limited effect on downstream predictions.

Reviewer wMoV

  • Section 3.3 explains when endpoint labels can miss beneficial partial injection, and Appendix D compares binary supervision with direct strength regression.
  • Section 4.4 holds gate initialization, row pooling, and checkpoint eligibility fixed across all three training variants.
  • Section 4.4 compares numerical-target AUROC using identical utility labels for both estimators. Appendix D reports AUROC and AUPRC under both frozen references.

We hope that the above revisions can further address your concerns. Thank you again for your diligent effort in reviewing.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.