acceptodds
Under review as a conference paper at ICLR 2027

EffSelect: Efficient Feature Value Selection for Deep Recommender Systems with Mini-Batch Training

Abstract

Features are critical to the performance of deep recommender systems, where they are typically represented as low-dimensional embeddings and fed into deep networks for prediction. However, a major challenge remains unaddressed: the sparsity and long-tail distribution in feature data result in a large number of non-informative feature values. These redundant values significantly increase memory usage and introduce noise, thereby impairing model performance. Most feature selection or pruning methods operate at a coarse granularity, either selecting entire features or fields, while finer-grained methods require a large number of additional learnable parameters. These methods struggle to effectively handle pervasive redundant features. To address these issues, we introduce EffSelect, a novel framework for finer-grained selection method at the level of feature values. Unlike previous methods, EffSelect directly quantifies the contribution to the prediction loss of each feature value as its importance. Specifically, we propose a mini-batch pre-training strategy that requires only 5% of the data for rapid warm-up, enabling real-time adaptation in dynamic systems. Using the trained model, we introduce an efficient and robust gradient-based mechanism to evaluate feature value contribution, discarding those features with low scores. EffSelect is theoretically guaranteed and achieves superior performance without introducing any additional learnable parameters to the base model. Extensive experiments on benchmark datasets validate the efficiency and effectiveness of EffSelect.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Feature Value Selection, Recommender Systems, CTR Prediction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Az6JOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EffSelect, a feature-value selection framework for deep recommender systems that aims to reduce redundant feature values and the corresponding embedding parameters while maintaining prediction performance. The framework first uses MFCS to select a small subset of training data for efficient pretraining, and then applies FeatIS to estimate feature-value contributions through a gradient-based mechanism. The most important feature values are retained for model retraining.

Critical strengths
  1. The method enables efficient feature value selection without additional learnable parameters.
  2. Experiments on four datasets demonstrate competitive performance with fewer model parameters.
Critical weaknesses
  1. The paper defines feature-value importance in Eq. (7) as the loss change caused by removing a feature value. However, the proposed method then uses Taylor expansion and integrated gradients to estimate importance scores, as shown in Eqs. (8)–(13). The connection between these estimated scores and the loss-change definition in Eq. (7) is not sufficiently justified. This is consequential because the validity of the feature-value selection criterion depends on whether these scores faithfully approximate the intended importance measure.
  2. The empirical evaluation is limited to a single feature-value selection ratio, α=10%\alpha=10\%, and does not provide a comprehensive comparison with baseline methods across different selection ratios. Evidence based on such a narrow setting is not particularly convincing. In addition, EffSelect underperforms baseline methods on the Avazu dataset, raising concerns about its generalizability and robustness across different types of recommendation datasets.
Questions for authors
  1. Could the authors clarify how well the estimates in Eqs. (8)–(13) approximate the actual loss changes defined by direct feature-value removal in Eq. (7)?
  2. Could the authors justify evaluating only a single feature-retention ratio, or provide any existing sensitivity results across different ratios? Additionally, could the authors explain why EffSelect underperforms relative to other methods on the Avazu dataset?
  3. The paper aims to satisfy both Proposition 1 (Distribution Consistency) and Proposition 2 (Feature Coverage Maximization). Given the long-tail distribution of the data, however, greedy feature-coverage maximization may disproportionately select samples containing rare tail features, creating a discrepancy between the selected-sample distribution and the original data distribution. How can Proposition 1 still hold under this greedy feature-coverage objective?
  4. The authors characterize the proposed method as both efficient and robust, but the experimental section provides no robustness analysis to substantiate the robustness claim. Could the authors clarify what evidence supports the claimed robustness? In addition, are any statistical significance tests available to support the reported superiority of the proposed method?
  5. The evaluation uses only a single feature-value retention ratio, α=10%\alpha=10\%. Could the authors clarify why this ratio was chosen and whether EffSelect remains effective under different feature-value retention ratios? In particular, do its relative advantages over baseline methods persist when the retention ratio is varied?
Reviewer_QMg4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a framework, EffSelect for fine-grained feature value selection in deep recommender systems. The authors contend that existing methods suffer from low efficiency and poor robustness, and so their framework addresses these via two components, maximizing feature coverage sampling (MFCS) and feature value importance scorer (FeatIS). MFCS is a greedy, bitmap-accelerated batch selection scheme, and FeatIS caters after pretraining the embedding table and model on the selected batches. Experiments on four benchmarks with DCN and MaskNet backbones, comparing with series of baselines such as field level methods, a value level gate method i.e. OptFS, and autoencoder based methods.

Critical strengths
  • The problem setting and the training-free, gate free design are coincidental. The authors correctly identifies a real problem in industrial recommender systems, feature value selection at tens of millions scale cannot afford per-value learnable gates nor repeated full pretraining, and gate embedding co-optimization creates competing objectives. It is a simple idea with a efficiency anecdote.

  • The theoretical scaffolding is used appropriately to strengthen the explanation. The submodularity proof for coverage maximization and the quadrature error analysis, midpoint under Lipschitz derivative assumptions are correct and well matched to the algorithmic choices.

Critical weaknesses
  • The gains are marginal and inconsistent. In Table 2, the framework improvements over the strongest baselines are often within noise, such as Criteo-DCN AUC 0.8102 vs. Base 0.8090, Ali-CCP-DCN 0.6021 vs. AdaFS 0.6004, iPinYou-DCN 0.7699 vs. 0.7683. It lacks confidence intervals, significance tests, and multi seed variance. Also, the comparison protocol is asymmetric, field-selection baselines are evaluated at the point where cumulative field importance first exceeds 10%, whereas value selection methods keep top 10% of feature values. These are different budgets in different units, so the shown superiority of value level over field level selection is confounded by the retention budget.

  • The robustness claim is weak, and the method introduces new hyperparameters it does not fully analyze. The paper motivates EffSelect by opposing gate based methods for hyperparameter sensitivity, yet the proposed method itself introduces ρ\rho, N, and α\alpha, and the α\alpha study shows opposite trends across datasets. On Criteo, performance peaks at small α\alpha then declines, while on Ali-CCP it rises monotonically to ~60% before degrading. On the other side, the zero-like vs. field-wise mean starting point i.e. EffSelect_Z vs. EffSelect_M shows different winners across datasets indicating the baseline choice is itself a sensitive knob.

  • The time complexity analysis lacks the cost of the N forward and backward passes over the validation set as a function of ∣V∣|V|, which is a bottleneck. In Fig. 3, comparison mixes per epoch and total times, which is confusing. The 10% features yields better than all features result on Criteo is interesting but unexplained, if 90% of values are removable with improvement, the attribution may be capturing regularization, which needs justification.

Questions for authors
  • The authors should add a comparison where field selection and value selection methods are matched on retrained parameter count.
  • Given the small deltas in Table 2, can you provide mean and std over at least 3–5 seeds and a paired significance test against the strongest baseline (OptFS/AdaFS) per dataset?
  • In Fig. 5–6, the authors show dataset dependent optima for α\alpha. How do you reconcile this with the stated robustness advantage over gate based methods?
  • EffSelect_Z and EffSelect_M trade wins across datasets. Is there a criterion for selecting the starting point a priori, or is this effectively another tuned hyperparameter?
  • Why does 10% beat 100% on Criteo? Is the gain from noise removal or regularization, or from de-noising of uninformative values? An analysis correlating removed value importance scores with value frequency and OOV rate would be interesting to see.
  • MFCS guarantees distribution consistency within the sampled batches but importance is computed on a validation set that may drift from pretraining data. How robust is the ranking under temporal or distribution shift?
3 comments
AuthorsOpenReview ↗

Sincere thanx again!

Dear Reviewer,

Thank you for your constructive reviews, which have helped us strengthen our paper. As you may appreciate, your rating is particularly important at this stage. With the discussion ddl approaching, we would like to ask whether our current response is convincing enough for you to support our work. If there are any further points for discussion, please feel free to leave a comment.

Best,

The Authors of EffSelect

Reviewer_QMg4OpenReview ↗

Official Comment By Reviewer

Thanks for providing the detailed analysis and clarifications. However, this party convinces me due to lack of FeatIS derivation in Eq. 11 with the error bound, and unclear definition and construction of Ev∗E_v*. On another note, the four dataset results table and the efficiency comparison vs OptFS still remains. So, I am inclined to keep my current score.

AuthorsOpenReview ↗

Sincere thanks.

Dear Reviewer,

We are pleased to see your follow-up. We will shortly provide new supplementary information addressing the points you raised, and we look forward to continuing our discussion with you.

Best regards

Reviewer_8LLeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EffSelect, a feature-value selection framework for reducing embedding redundancy in deep recommender systems. The framework consists of two components: Maximizing Feature Coverage Sampling (MFCS), which greedily selects batches covering diverse feature values for pretraining, and Feature Value Importance Scorer (FeatIS), which estimates importance using gradient-based attribution along a path from reference embeddings to pretrained embeddings. The reference embeddings are either zero vectors or field-wise means. Feature values are ranked by their scores, and the model is retrained on the full training set using the retained values, with discarded values mapped to OOV embeddings. The default configuration uses 5% of the training data for pretraining and retains 10% of feature values. Experiments on four CTR datasets with DCN and MaskNet demonstrate substantial parameter reductions and competitive or improved predictive performance on three datasets, while performance declines on Avazu.

Critical strengths

1.The paper addresses a practically relevant problem. Individual feature values within the same field can differ substantially in their predictive utility, making field-level selection overly coarse. Selecting individual embedding rows provides finer control over the parameter budget of large-scale recommender models while retaining useful values within each field.

2.The framework avoids additional learnable gating parameters and is readily applicable to different backbone models. EffSelect uses the embeddings and gradients of the underlying recommender model to estimate feature-value importance, without requiring a separate selection network or substantial architectural modifications.

Critical weaknesses

1.The claim that MFCS preserves the original data distribution is insufficiently justified. Randomly partitioning the dataset into batches without replacement does not guarantee that the subset subsequently selected through coverage maximization follows the original distribution. The selection criterion favors batches containing more distinct feature values, which may alter the representation of long-tail values, label proportions, and feature co-occurrence patterns.

2.The methodological novelty appears limited. The main contribution lies in combining established ideas and adapting them to feature-value selection in recommender systems. While this integration may offer practical benefits, the manuscript does not sufficiently clarify what new capabilities or technical insights it provides beyond closely related approaches.

3.The ablation studies do not adequately isolate the contributions of the core components. The current experiments show that the complete pipeline outperforms random feature-value retention and direct prediction using a model pretrained on a small data subset. However, these comparisons change multiple aspects of the pipeline simultaneously and therefore do not isolate the contribution of MFCS.

4.Several inconsistencies in the equations, pseudocode, and experimental descriptions hinder reproducibility. For example, the manuscript states that Eq. (13) reduces to Eq. (9) when N=1, but the gradients in these expressions are evaluated at E and E*, respectively. Appendix D.3 also specifies a feature retention ratio of 50%, whereas the main experiments use 10%.

Questions for authors

1.Novelty relative to the closest alternative. Could the authors clarify the main technical contribution of the proposed method beyond pretraining on a small random subset of the data and then directly applying integrated gradients to select feature values?

2.Distributional bias introduced by MFCS. After coverage maximization, how much do the selected subset’s label proportions, feature-value frequencies, and long-tail distribution differ from those of the original training set?

3.Isolating the contribution of MFCS. The current ablation directly uses the model pretrained on a small data subset for prediction, whereas the complete method additionally performs feature-value scoring, selection, and retraining on the full training set. Could the authors keep all other steps identical, replace MFCS with random sampling at the same data ratio, and compare the final performance?

Comments on the paper

2 comments
Reviewer_8LLeOpenReview ↗

Official Comment by Reviewer 8LLe

Thank you for the additional experiments and corrections. However, the empirical distributional analyses do not establish the claimed distribution-preservation guarantee, and my concern about limited methodological novelty remains. I therefore maintain my rating.

Reviewer_8LLeOpenReview ↗

Official Comment by Reviewer 8LLe

Thank you for the additional comparisons and clarifications. The random-sampling results alleviate my empirical concern about distributional distortion, but do not establish the theoretical preservation guarantee. My concern about the limited methodological novelty also remains. I maintain my rating of 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.