acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Feature Evolution Learning with Multi-Label Evidence Transfer: Algorithm and Theory

Abstract

Multi-label classification (MLC) conventionally assumes a stationary feature space, whereas real-world data streams often undergo simultaneous feature disappearance and emergence, disrupting label-discriminative evidence and rendering historical label dependencies increasingly mismatch. To address this chanllenges, we propose AFET, a unified framework that recasts dynamic-feature MLC as preserving and transferring label-relevant evidence across evolving feature spaces. Specifically, AFET exploits survived features as a cross-stage semantic anchor, recovering information associated with vanished features while preserving historical discriminative semantics through label-conditioned latent alignment. It further integrates recovered, survived and augmented evidence through instance-adaptive transformations, while adapting evolving label dependencies via structure-aware distillation and dynamically calibrating label-wise decision thresholds online. Theoretical analysis characterizes prediction stability across feature recovery, discriminative alignment, structural transfer and threshold adaptation. Extensive experiments on seven benchmarks under both one-shot and multi-shot settings demonstrate consistent improvements over competitive baselines.

Primary area:
learning on time series and dynamical systems
Keywords:
Multi-label Classification, Feature Evolution, Streaming Learning, Label Dependency Modeling, Prediction Calibration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Kh8qOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on multi-label classification in real-world data streams where features may simultaneously disappear and emerge, disrupting label-discriminative information and making previously learned label dependencies less reliable. To address this challenge, the authors propose AFET, a framework designed to preserve and transfer historical inter-label dependencies in a one-pass learning setting without replaying historical samples. Specifically, AFET leverages surviving features to reconstruct information associated with vanished features and employs Label-Conditioned Latent-Space Alignment to preserve label-discriminative information. It further incorporates Structure-Aware Distillation, an evolving label graph, and Adaptive Prediction Calibration to integrate historical knowledge with augmented features. The authors also provide a theoretical analysis of prediction stability and conduct comparative and ablation experiments on seven benchmark datasets. The experimental results demonstrate competitive performance across multiple evaluation metrics, while the ablation studies further illustrate the contributions of the main components.

Critical strengths
  1. Unified Framework for Cross-Stage Evidence Transfer. AFET uses surviving features as a bridge between learning stages, integrating vanished-feature recovery, label-conditioned latent-space alignment, structure-aware distillation, an evolving label graph, and adaptive prediction calibration into a unified framework. The method considers not only feature representations but also the adaptation of label dependencies and decision thresholds as the feature space evolves. The motivations behind the individual components are generally clear, and their integration reflects a clear approach to addressing feature evolution in multi-label classification.
  2. Preservation of Label-Discriminative Information. The paper investigates multi-label classification in data streams where features can simultaneously emerge and disappear. An important observation is that accurate feature reconstruction does not necessarily guarantee the preservation of label-discriminative information. Building on this observation, the authors further consider mismatches in historical label dependencies and potential shifts in prediction-score distributions under feature evolution. This perspective goes beyond feature reconstruction alone and offers a meaningful extension to existing studies on feature evolution, with practical relevance to real-world data streams. I consider this a strong strength because it connects the method’s central motivation to both a specific design choice and empirical evidence of its contribution to classification performance.
Critical weaknesses
  1. Insufficient Justification for the Threshold Calibration Strategy. In Online Label-Wise Threshold Calibration, the target positive ratio is primarily determined by the model's own predictions. However, the authors do not sufficiently explain why increasing this ratio should lead to better classification decisions. In particular, it remains unclear whether the reported recall improvements reflect genuinely better classification performance or mainly result from predicting more positive labels, potentially at the cost of increased false positives.
  2. Unclear Independent Contributions of the Core Mechanisms. AFET integrates multiple techniques into a unified framework, but the necessity and independent contributions of some core mechanisms are not sufficiently validated. Although the existing ablation studies provide useful evidence, they mainly compare broad modules or different information sources. The individual contributions of structure-aware distillation and the evolving label graph are not fully established by the current ablations. This makes it difficult to assess the novelty of the proposed evidence-transfer design and whether the additional complexity introduced by these mechanisms is justified by their performance gains.
Questions for authors
  1. In Section 2.4, the authors introduce Online Label-Wise Threshold Calibration to adapt the decision threshold for each label. However, the motivation for the target positive ratio remains insufficiently explained, particularly why the updated ratio should lead to better classification decisions than the previous one. Specifically, Eq. (8) first estimates the current pseudo-positive rate using the previous threshold: [ \hat{\pi}{t,l}=\frac{1}{n_t}\sum{i=1}^{n_t}\mathbb{I}[P^S_{t,il}>\tau_{t-1,l}]. ] The target positive ratio is then defined as [ r_{t,l}=\operatorname{clip}((1+\rho)\hat{\pi}{t,l}+m,r{\min},r_{\max}), ] followed by a quantile-based threshold update with exponential smoothing: [ \tilde{\tau}{t,l}=\operatorname{Quantile}({P^S{t,il}}{i=1}^{n_t},1-r{t,l}), ] [ \tau_{t,l}=\gamma\tau_{t-1,l}+(1-\gamma)\tilde{\tau}{t,l}. ] The target ratio is therefore determined primarily by the model's own predictions, without directly accounting for the true positive label prevalence or classification errors. It remains unclear why increasing the predicted positive ratio should necessarily improve classification performance. More specifically, the update rule may continuously lower the decision threshold even when the prediction score distribution remains unchanged. Consider a fixed model with a stationary score distribution whose cumulative distribution function (CDF) is continuous and strictly increasing. Following the definition in Theorem 3.5, we have [ \pi^\star{t,l}=1-F_{t,l}(\tau_{t-1,l}). ] When clipping is inactive, and assuming the adjustment terms produce a strictly positive increase, the target ratio satisfies [ r^\star_{t,l}=(1+\rho)\pi^\star_{t,l}+m>\pi^\star_{t,l}. ] Consequently, the corresponding target threshold satisfies [ \tau^\star_{t,l}=F_{t,l}^{-1}(1-r^\star_{t,l})<\tau_{t-1,l}. ] Under this population setting, the exponential smoothing update also decreases the threshold. This suggests that the calibration rule can introduce systematic threshold changes even in the absence of distribution drift, rather than responding solely to changes in the underlying data distribution. The authors should provide further justification for the target-ratio update rule and clarify how its induced threshold changes relate to actual classification performance.
  2. In Table 3 and Table 6, the authors mainly conduct ablation on the whole Structure-Aware Evidence Transfer(M2M_2). Although these results show that M2M_2 can improve the performance, the individual contributions of Evolving Label Graph and Structure-Aware Distillation are still not clear. Figure 2 has already compared the performance of static graph and evolving graph, but a more systematic comparison of graph evolution and distillation mechanisms under the same experimental settings is still missing. Could the authors further compare fixed Historical Graph and Evolving Label Graph while keeping other modules, teacher/student networks, and training settings unchanged? Under each graph setting, could the authors also compare Response Distillation Only and Response + Structure Distillation? These comparisons would help clarify which mechanism mainly contributes to the performance improvement of M2M_2.
1 comment
AuthorsOpenReview ↗

Rebuttal by Authors

We sincerely appreciate the reviewer for the valuable feedback. We respond to your concerns as follows:

A1. We clarify the motivation, theoretical scope, and empirical effectiveness of our calibration strategy. 1) Motivation. Under feature evolution, the disappearance of informative features may reduce positive-label confidence, making fixed thresholds less appropriate. Our quantile-based calibration adapts label-wise thresholds to current score distributions, while ρ\rho and mm introduce a recall-oriented adjustment to mitigate missed positives. The target positive ratio is intended to control the decision operating point, rather than estimate the true label prevalence. Clipping and exponential smoothing constrain the target ratio and update speed, respectively. 2) Theoretical clarification. We agree that under a stationary continuous score distribution FlF_l, when rt,l>πt,lr_{t,l}>\pi_{t,l} and clipping is inactive, the target threshold satisfies τt,l~=Fl−1(1−rt,l)<τt−1,l.\tilde{\tau_{t,l}}=F_l^{-1}(1-r_{t,l})<\tau_{t-1,l}. Thus, threshold reduction may occur even without distributional drift, reflecting the intended recall-oriented adjustment. Importantly, Theorem 3.5 establishes tracking stability relative to the prescribed target quantile, rather than classification optimality or guaranteed improvements in precision. We will clarify this distinction and its limitations in the revision. 3) Empirical evidence. To isolate the effect of threshold calibration, we compare three strategies using identical prediction scores, which are Fixed (τ=0.5\tau=0.5), Unadjusted (ρ=m=0\rho=m=0), and Ours. All other components remain unchanged.

DatasetStrategyF1 ↑\uparrowRecall ↑\uparrow
EnronFixed0.7290.700
Unadjusted0.7350.720
Ours0.7530.790
NUS-WIDEFixed0.6660.620
Unadjusted0.6740.650
Ours0.6860.740

Compared with Unadjusted, Ours improves F1 and Recall on Enron and NUS-WIDE, respectively. These results indicate that the recall gains are accompanied by improved overall classification performance, rather than merely increasing positive predictions without improving F1. While this does not imply fewer false positives, it demonstrates a more favorable trade-off. We will also clarify that the existing M3M_3 ablation evaluates the complete adaptive prediction module, not threshold calibration alone.

A2. The existing M2M_2 ablation evaluates the complete structure-aware evidence transfer module without fully isolating its core mechanisms. 1) Mechanism clarification. The two mechanisms address complementary challenges. Evolving Label Graph (Eq. 4)integrates historical dependencies with current prediction-induced relationships to mitigate structural mismatch under feature evolution. Structure-Aware Distillation (Eq. 5) transfers both label-wise responses and inter-label dependencies from the frozen teacher to the student. Thus, graph evolution adapts relational representations, while structure-aware distillation preserves historical relational knowledge. 2) Controlled ablation. Following the reviewer's suggestion, we consider four configurations by varying the graph strategy (Fixed Historical vs. Evolving) and distillation objective (Response Only vs. Response + Structure), while keeping all other modules, teacher/student architectures, and training settings unchanged. 3) Results and analysis. Both mechanisms provide consistent improvements. Their combination achieves the best performance on both datasets, demonstrating complementary benefits rather than gains attributable to a single mechanism. Moreover, graph evolution and structural distillation reuse existing prediction representations and introduce no additional trainable networks, supporting a lightweight integration. We will incorporate these findings and clarify the individual contributions in the revision.

DatasetGraphDistillationAP ↑\uparrowRecall ↑\uparrow
EnronFixedResponse Only0.780.74
FixedResp. + Struct.0.810.76
EvolvingResponse Only0.820.77
EvolvingResp. + Struct.0.850.79
NUS-WIDEFixedResponse Only0.690.65
FixedResp. + Struct.0.700.69
EvolvingResponse Only0.710.70
EvolvingResp. + Struct.0.720.74
Reviewer_VxaZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper focuses on the task of multi-label classification where the number of classes can grow. The paper introduces AFET, a unified framework for multi-label learning under feature evolution. It is designed to preserve historical predictive knowledge while incorporating newly available features. It uses survived features as the bridge between historical and evolving feature spaces. Before evolution, a teacher learns to recover vanished features from survived ones, aligns latent representations conditioned on labels, and encodes historical label dependencies as a structural prior. After evolution, a student combines all features using adaptive-depth representations. The method has been extensively evaluated on many independent datasets and outperforms all the baselines on all datasets.

Critical strengths
  • Strong results on all datasets.
  • Sufficient theoretical justification for the decisions made in the paper.
Critical weaknesses
  • In context learning methods seem to be missing in the results table. In in-context learning, new labels can be easily described and added. For example, ICXML seems to be useful in these settings.
Questions for authors

None.

1 comment
AuthorsOpenReview ↗

Clarification on In-Context Learning Baselines

We sincerely thank your valuable feedback. We agree that in-context learning (ICL) methods, such as ICXML (Zhu and Zamani, 2024), offer promising capabilities for handling previously unseen labels. However, we would like to clarify that ICXML and our AFET address fundamentally different challenges in multi-label learning.

(1) Different problem settings. ICXML focuses on zero-shot extreme multi-label classification, where candidate labels can be identified and predicted through language-based contextual information. In contrast, AFET investigates multi-label learning under simultaneous feature emergence and disappearance, where the label space remains fixed, but the observable input features continuously evolve. As formally defined in Section 2.1, our objective is to preserve and transfer historical label-discriminative evidence when previously available features become inaccessible.

(2) Different adaptation challenges. The ability to introduce new labels through textual descriptions does not directly address the loss of predictive information caused by disappearing input features. AFET explicitly tackles this challenge through survived-to-vanished feature recovery, label-conditioned representation alignment, and structure-aware knowledge transfer. Moreover, our setting follows a one-pass streaming protocol in which historical instances cannot be revisited, whereas ICXML is not specifically designed to preserve predictive evidence across evolving feature spaces.

(3) Baseline comparability. Our experiments therefore prioritize methods targeting feature evolution and online multi-label learning. Directly applying ICXML would require additional adaptations to accommodate evolving feature representations, streaming constraints, and non-textual datasets. Consequently, an unmodified comparison would not evaluate both methods under equivalent assumptions.

We appreciate the reviewer for highlighting this relevant research direction. We will clarify the distinction between feature-space evolution and label-space adaptation in the revised manuscript and discuss the potential integration of ICL techniques for future extensions.

Reviewer_7bTYOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses multi-label classification (MLC) in a one-pass stream with an evolving feature space: the historical stage observes vanished (V) and survived (S) features, the new stage observes S and augmented (A) features — the OPID setting (Hou & Zhou, TPAMI 2017) extended to multi-label, as in the recent MLID (Jiang et al., AAAI 2026). The proposed AFET has three parts:

  1. Historical teacher (Sec. 2.2, A.2.1): a VAE on X^(s) that reconstructs X^(s) and X^(v) (S→V recovery); a discriminative encoder on [X^(v), X^(s)]; a label-conditioned prototype alignment loss L_LCA (Eq. 1) between the two representations; a co-occurrence label graph used for logit propagation (Eq. 2).
  2. New-stage student (Sec. 2.3, A.2.2): inherits the VAE/decoder, builds the input [X̂^(v), X^(s), X^(a)], fuses the historical graph with a prediction-induced graph (Eq. 4), and distills teacher probabilities and normalized P^T P co-occurrence matrices (Eq. 5). The teacher's targets are computed on its own recovered features (Eq. 18).
  3. Adaptive prediction (Sec. 2.4, A.2.3): soft mixture over residual blocks of different depth with a depth regularizer (Eq. 6, 19), and an unsupervised label-wise threshold update based on pseudo-positive rates and batch quantiles with EMA (Eq. 8).

Sec. 3 / A.3 give Lipschitz-type bounds: loss difference controlled by recovery error (Thm 3.1), label-region prediction discrepancy controlled by prototype distance plus dispersion (Prop 3.2), co-occurrence discrepancy controlled by the KD loss (Prop 3.3), graph-fusion perturbation (Prop 3.4), a DKW-based threshold tracking bound (Thm 3.5), and a "conditional" Rademacher generalization bound (Thm A.13). Experiments use seven standard small/medium MLC benchmarks with feature evolution simulated by splitting feature columns into contiguous blocks. AFET is compared with eight baselines (FLLS, FESL, OPID, OLD3S, MLID, OKS-OMC, CLIF, SLDL), using AUC tables, bar plots for other metrics, scenario/module ablations, and sensitivity grids.

Critical strengths
  • The problem (multi-label learning under simultaneous feature disappearance and emergence) is practically relevant, and the paper correctly points out that pure feature-reconstruction objectives (as in FESL, OPID, OLD3S) need not preserve label-discriminative information. Motivating Fig. 1–2 make the point clearly.
  • Using the teacher's own recovered features to produce distillation targets after V becomes inaccessible (Eq. 18) is a sensible, leak-free design choice, and the scenario ablation protocol (A.4.2) is described carefully (masking blocks with zeros, no use of historical labels in no-history settings).
  • The scenario ablation (Tables 2, 5) is a useful kind of experiment: it separates the value of history, recovered V, and new A.
  • Evaluation covers several ranking and thresholded metrics, both one-shot and multi-shot evolution, and seven datasets.
Critical weaknesses

1. Empirical results contain multiple red flags that undermine trust in Table 1 and the ablations.

  • Below-chance AUC for learned baselines. In Table 1, FLLS (e.g., .4383, .4996), OLD3S (.4366 on Languagelog, .4893 on Enron, .4859/.4865 on Rcv1v2) and FESL (.4794–.4845 on Tmc2007) have macro-AUC below 0.5, i.e., worse than random ranking, which raises concerns about the evaluation. Could these results reflect implementation or adaptation issues, such as label/sign flips, score orientation, or degenerate outputs? Please verify the evaluation and report a simple model on S alone as a reference.
  • Identical numbers across settings. Many baselines report exactly the same mean and standard deviation for every n_i: OLD3S .4366 (.0070) in all four Languagelog rows, .4893 (.0153) on Enron, .6251 (.0097) on Bibtex; OKS-OMC .7921 (.0013) and .8059 (.0042); SLDL constant on every dataset; FLLS .5305 (.0186) for three Rcv1v2 rows and .5238 (.0928) for three Tmc2007 rows. The paper says CLIF/SLDL/OKS-OMC are "evaluated only at the post-incremental stage", but that does not explain why OLD3S (a streaming method) or FLLS would give identical outputs to four decimals under different batch sizes.
  • Repeated table entry requiring clarification. CLIF's entry for Bibtex n_i=700, ".7942 (.0061)", appears again as CLIF's NUS-WIDE n_i=500 entry, with the same standard deviation.
  • Inconsistent AP across tables/figures. With the full model, Table 2 gives AP = 0.85 (Enron), 0.72 (NUS-WIDE), 0.85 (Tmc2007), and Table 5 gives 0.75 (Languagelog), 0.65 (Bibtex), 0.64 (Rcv1v2). The sensitivity grids for the same one-shot setting give AP ≈ 0.737–0.752 for Enron (Fig. 6a), ≈ 0.418–0.431 for NUS-WIDE (Fig. 9e), ≈ 0.613–0.621 for Languagelog (Fig. 9a), ≈ 0.42–0.49 for Rcv1v2 (Fig. 9d), and up to 0.96 for Bibtex (Fig. 9c). No hyperparameter in those grids comes close to the table values on several datasets. Which configuration is the reported model?
  • Variance and significance. The number of runs/seeds is never stated, no significance tests are reported, and several "wins" are within one standard deviation (e.g., Scene n_i=60: .9204 vs CLIF .9201; Tmc2007 n_i=400: .8543 vs .8518; Bibtex n_i=700 has a standard deviation of .0337). Together, these make it impossible to conclude that AFET "consistently outperforms" the baselines.

2. The threshold calibration (Eq. 8) appears to have a degenerate fixed point, and its theory tracks a self-referential target. If the score distribution is roughly stationary and τ_{t-1} is close to the previous target quantile, then π̂_t ≈ 1 − F(τ_{t-1}) ≈ r_{t-1}, so r_t ≈ clip((1+ρ) r_{t-1} + m). Under this approximation, with m > 0 and before clipping, the recurrence has geometric growth when ρ > 0 and linear growth when ρ = 0. Does the update saturate at r_max under the actual score dynamics and parameter settings, causing predicted positive rates to approach the same limit across labels? This never uses the labels Y_t, even though they are observed in the training stream. It would explain the large recall jump in the module ablation (Table 3: NUS-WIDE recall 0.40 → 0.74 with M3; Table 6: Rcv1v2 0.39 → 0.87) without any real calibration gain, since recall can be raised trivially by lowering thresholds. Precision and F1 (micro/macro) are not reported, which would expose this. Thm 3.5 does not help: the "ideal" τ*{t,l} is defined by the same heuristic rule applied to the population (A.3.6), so the bound only shows that the algorithm tracks its own update rule, not any optimal (e.g., F1- or HL-optimal) or calibrated threshold. The drift Δ{t,l} is itself caused by the algorithm. The DKW step also assumes scores are i.i.d. given the history, but thresholds are computed on the same mini-batch the student was just trained on (A.2.3), so that conditioning is violated. Values of ρ, m, r_min, r_max, γ are not reported.

3. The theory consists of trivial, mostly unverifiable or possibly vacuous bounds, and does not match the algorithm. The title promises "Algorithm and Theory", but:

  • Thm 3.1 is the composition of the 1/√(nL)-Lipschitz BCE (Lemma A.1) with an assumed local Lipschitz constant K_t of a deep network. K_t is uncontrolled and unestimated. The RHS ‖X̂^(v),S − X^(v)‖ is unobservable in the new stage. In addition, after evolution the vanished decoder is fine-tuned with no supervision on X^(v) (Eq. 17), so nothing in the algorithm keeps that term small. The bound also compares two inputs to the same student; it does not say the student is good.
  • Prop 3.2 / Thm A.4 is a triangle inequality. Its RHS includes the within-label dispersions s^s, s^vs, which the paper admits are not optimized (lines 949–952). It also applies to the teacher's two representations in the historical stage, not to the student that makes the reported predictions. The appendix even redefines H_vs as the "survived-plus-recovered pathway" (line 919), contradicting the main text (complete [V,S] encoder, line 169).
  • Prop 3.3 / Lemma A.2: K_N = 1/ε + G_max/ε², where ε is the small stabilizer in Norm(·). With the usual ε (e.g., 1e-8) the second inequality, ‖R^S−R^T‖² ≤ 4K_N² n²L² L_KD, is numerically vacuous. Since Norm(M) = M/‖M‖_2 is scale-invariant, a meaningful bound would depend on ‖M‖_2 rather than on ε. L_dep in Eq. 42 is not defined in the main text.
  • Prop 3.4 is ‖QA‖_F ≤ ‖Q‖_2‖A‖_F. Cor. A.8 assumes the historical graph error A_hist − A* is zero-mean and uncorrelated with the new-graph error. But the paper's own motivation (Fig. 2, "increasingly mismatched") is that A_hist is systematically biased with respect to A*, so the assumption contradicts the premise. The schedule actually used for η_t is never specified, so it is unclear whether the "optimal" η_t* is used at all.
  • Thm A.13: the constant B is undefined. The 1/√(n_t L) factor in the Rademacher term is unusual and not derived. The bound is uniform over a fixed graph A, while A_t is data-dependent (built from predictions on the same batch). Assumption A.11 conditions on mappings learned from the same samples. The bound controls the oracle risk R*_t (oracle representation h*) by the empirical risk on recovered features, whereas the quantity of interest is the risk of the deployed predictor. Leftover references to "Theorem S2/S3" (lines 1113, 1193) suggest the appendix was assembled from another manuscript. In short, none of the results is a regret or generalization guarantee for AFET as run. They restate that "if each component's error is small, the output changes little", which is true of any Lipschitz pipeline. By comparison, FESL (Hou et al., NeurIPS 2017) and OPID give regret/ensemble guarantees for the actual learner.

4. Limited novelty: AFET is a combination of known components with a modest delta over MLID/OPID/OLD3S. S→V recovery through a VAE/shared latent space is the core of OLD3S (Lian et al., TKDE 2023). Compressing vanished information into survived features is OPID/MLID. Label-prototype alignment is standard (class-conditional/prototype alignment in domain adaptation). Distilling Gram/relational matrices is relational KD (e.g., Park et al., CVPR 2019) or similarity-preserving KD (Tung & Mori, ICCV 2019). Logit propagation over a co-occurrence graph follows ML-GCN-style label-graph methods (Chen et al., CVPR 2019). Instance-adaptive depth mixing is the hedge-backprop / OLD3S idea (Sahoo et al., IJCAI 2018). Quantile thresholding is a simple heuristic. Each is reasonable, but the paper does not identify a new principle, and the ablations do not isolate the multi-label-specific parts (e.g., the dependency term ω‖R^S−R^T‖ vs response KD alone, or a static vs fused graph) in the main experiments. The "evidence" terminology is used informally and is unrelated to evidential deep learning (Sensoy et al., NeurIPS 2018), which may mislead readers.

5. Experimental design is weak relative to the claims.

  • Feature evolution is simulated by cutting the original feature columns into equal contiguous blocks (Sec. 4.1) on small benchmarks (Languagelog/Enron/Scene with ≤ ~2.4k instances). No real evolving-sensor stream is used, and no random or adversarial block assignments are tested.
  • Table 4 dimensions look inconsistent with standard versions of the datasets: Rcv1v2 subset1 is listed with d_v+d_s+d_a = 25+51+25 = 101, the same as its label count, whereas the standard version has ~47k features; Bibtex sums to 1186 vs the standard 1836. The "first-stage sample counts" (e.g., NUS-WIDE 134,824) are also unclear relative to the stated n_i. Preprocessing must be stated.
  • How single-label/binary methods (FESL, OPID, OLD3S, FLLS) were adapted to multi-label is not described (binary relevance? one network per label?), and it is unclear whether these adaptations explain any of the below-chance AUCs. Please document and verify the adaptation.
  • Missing strong simple baselines: (a) the same student network trained on [S, A] only, i.e., scenario S2, which in Table 2 is often within 0.01–0.03 AP of the full model (Enron 0.82 vs 0.85, Tmc2007 0.83 vs 0.85); (b) mean/zero imputation of V plus the same network; (c) an oracle with V available at test time as an upper bound. Other feature-evolvable-stream methods are not discussed, e.g., FDESL / feature and distribution evolvable streams (Zhang et al., ICML 2020) and capricious/trapezoidal-stream learners (OCDS, He et al., IJCAI 2019; Zhang et al., TKDE 2016). I am confident these works exist but less sure of every venue detail.
  • The headline "only 4% degradation vs FLLS 18%, FESL 32%, ..." (Sec. 4.2) has no stated metric or computation.
  • Reproducibility: K, hidden sizes, α, ω, η_t, ρ, m, r_min/r_max, γ, the number of local updates per batch, the learning rate, seeds and compute are not given, and no code is provided. "Uniform class distribution across all batches" (Sec. 4.1) is unexplained for multi-label data. The train/test protocol (prequential test-then-train vs a held-out test set) is not specified.

6. Protocol ambiguity around "one-pass". The teacher and student perform "a fixed number of local updates" per mini-batch (A.2.2). How many updates are used? If this number exceeds one, how does the resulting protocol compare with the one-pass OPID/MLID baselines, and is the update budget matched?

Minor Issues

  • Typos: "chanllenges" (abstract), "Evolviong" (Fig. 2), "Gragh" (Fig. 3), "showm", "Langugelog", "Rcv1v1(subset1)" vs "Rcv1v2", "Tme2007" (Fig. 11).
  • "Fig.(7)" is cited in Sec. 4.2 for main-text results, but Fig. 7 is in the appendix; Fig. 4 appears to be meant.
  • Table 1 caption says "different number of training and testing examples", while the column is batch size n_i.
  • Inconsistent reference "C Hou, 2017" for OPID (author list garbled); OLD3S is cited as both Lian et al. 2023 and Liu et al. 2022 (A.1.2, line 666).
  • Several citations are tangential or odd for the claims they support, e.g., "Fivush & Slackman 2026, The acquisition and development of scripts" for sensor acquisition constraints, and "Chow 2024" on 6G optical communication.
  • The main text says MLID is one of "the first three baselines" designed for feature evolution, but MLID is the fifth listed; the categorization in Sec. 4.1 is muddled.
  • "Theorem S2/S3" leftover labels; Props 3.2–3.4 in the main text are called Theorems A.4–A.7 in the appendix.
  • Eq. (1) prototype denominators and the definitions of L_dep and B should be given in the main text.

Justification of Rating

The problem is relevant and the overall design is reasonable, but the paper does not meet the ICLR bar on any of its three pillars. The algorithm is mostly a combination of established components with a modest delta over OPID/OLD3S/MLID. The "theory" consists of one-line Lipschitz/triangle-inequality statements whose RHS terms are unobservable, possibly vacuous (K_N ∝ 1/ε²), based on assumptions that contradict the paper's own motivation (zero-mean historical graph error), or self-referential (the threshold target). The experiments show several signs of unreliable baseline results and internal inconsistencies: below-chance AUCs, identical values across settings, a duplicated table entry, and AP mismatches between ablation tables and sensitivity grids. Key hyperparameters and seeds are also missing. I would raise my score substantially if the authors (i) fix and rerun baselines with documented multi-label adaptations and multiple seeds with significance tests, (ii) reconcile the table/figure inconsistencies, (iii) show that the calibration does more than drive positive rates to r_max (precision/F1, a comparison to supervised thresholding), (iv) add strong simple baselines (S2, imputation, oracle-V), and (v) either replace the stability results with guarantees about the deployed learner or reposition them as intuition rather than a main contribution. Without (i)–(iii), I cannot recommend acceptance.

Questions for authors
  1. Why do FLLS, OLD3S and FESL obtain macro-AUC < 0.5 on several datasets? Please verify score orientation and the multi-label adaptation, and report a trivial baseline (label-frequency prior, logistic regression on S) for reference.
  2. Why are many baseline means and standard deviations identical across n_i (e.g., OLD3S on Languagelog, Enron, Bibtex; FLLS on Rcv1v2, Tmc2007), and why does CLIF's Bibtex n_i=700 entry reappear exactly as NUS-WIDE n_i=500? Were baselines rerun per setting?
  3. Which configuration produced Tables 2/3/5/6? Their AP values (e.g., NUS-WIDE 0.72, Enron 0.85) are far from every cell of Figs. 6/9/10 (≈0.42–0.43 and ≈0.74–0.75).
  4. What are ρ, m, r_min, r_max, γ? Please plot r_{t,l} over time. Does it saturate at r_max as the fixed-point argument in Weakness 2 suggests? Please report precision, micro/macro-F1 and Hamming loss with and without calibration, and compare against a simple supervised per-label threshold tuned on the labeled stream.
  5. How many seeds were used, and are improvements statistically significant (e.g., paired tests across seeds/datasets)?
  6. How is η_t chosen in practice? Is Cor. A.8's η_t* used, and how are v_hist, v_new estimated?
  7. What is ε in Norm(·), and what is the numerical value of K_N (Lemma A.2) in your experiments?
  8. Since the vanished decoder receives no supervision in the new stage (Eq. 17), how does ‖X̂^(v),S − X^(v)‖ evolve after evolution? Please measure it on held-out ground truth V, to check whether Thm 3.1's RHS stays small or drifts.
  9. How do the module ablations decompose M3 into depth routing vs threshold calibration, and M2 into response KD vs dependency KD vs the fused graph?
  10. Please clarify the Table 4 feature dimensions for Rcv1v2 and Bibtex, and the train/test protocol.
1 comment
AuthorsOpenReview ↗

Rebuttal by Authors

We sincerely appreciate the reviewer for the valuable feedback. We respond to your concerns as follows:

A1. The original single-label baselines were evaluated using one-hot predictions, which restrict each instance to one positive label and discard continuous ranking information. This may partly explain the below-chance AUC. We will re-evaluate these baselines using continuous label-wise scores.

A2. All baselines were independently rerun for each dataset and batch-size setting. OKS-OMC and SLDL are independent of streaming batch sizes. OLD3S and FLLS exhibit dataset-dependent sensitivity to batch size, potentially yielding identical results at the reported precision. We apologize for the transcription error in Table 1. The correct CLIF result for Bibtex (nin_i=700) is 0.5466 (0.0100).

A3. The discrepancy arises from different model configurations. Tables 2/3/5/6 report the final tuned AFET, whereas Figs. 6/9/10 examine the sensitivity to selected hyperparameters while fixing others at potentially suboptimal values. We will explicitly report these configurations and clarify this distinction in the revision.

A4. We use ρ=0.2,m=0.02,rmin=0.01,rmax=0.5ρ=0.2, m=0.02, r_{min}=0.01, r_{max}=0.5, and γ=0.9γ=0.9. Our implementation applies dual EMA smoothing, subtracts m in the threshold space, and clips thresholds to [0.05,0.5], mitigating but not eliminating saturation. Using identical prediction scores, calibration improves F1/Recall from 0.729/0.700 to 0.753/0.790 on Enron and from 0.666/0.620 to 0.686/0.740 on NUS-WIDE, demonstrating an improved trade-off. Theorem 3.5 establishes conditional target-tracking rather than decision optimality.

A5. All results are averaged over 20 independent runs. Welch's t-tests on Table 1 show significant AUC improvements over the strongest observed baseline in 25/28 settings (p<0.05p < 0.05), with 20/28 remaining significant after Holm correction. We will report these results and moderate our performance claims accordingly.

A6. In practice, we fix η=0.2η=0.2 while dynamically updating the prediction-induced label graph. Corollary A.8 provides a theoretical characterization of the trade-off rather than an operational selection rule. Thus, neither η∗η* nor the error variances are explicitly estimated in our implementation.

A7. We use ε=10−8ε=10^{-8}. The original KN=108+1016GmaxK_N=10^{8}+10^{16}G_{max} is overly conservative. For the implemented row normalization, we derive a tighter bound ∥RS−RT∥F≤1+Ldmin⁡∥GS−GT∥F\Vert R^S-R^T\Vert_{F}\leq\frac{1+\sqrt{L}}{d_{\min}}\Vert G^S-G^T\Vert_{F}, where dmind_{min} is the minimum εε-regularized row sum across both Gram matrices after removing self-loops. This avoids explicit ε−2ε^{-2} dependence.

A8. The recovery pathway inherits historical knowledge and is subsequently guided by classification and distillation, targeting predictive evidence preservation rather than exact feature reconstruction. Theorem 3.1 provides a conditional stability bound, not a guarantee of bounded recovery error. We will track reconstruction errors on held-out vanished features used exclusively for evaluation, without exposing them to the learner.

A9. Controlled M2M_2 ablations show that graph evolution and dependency KD independently improve AP by 0.04/0.03 on Enron and 0.02/0.01 on NUS-WIDE, respectively. Their combination achieves the best AP (0.85/0.72), demonstrating complementary benefits without additional trainable networks. For M3M_3, A4 isolates threshold calibration using identical prediction scores.

A10. We use the first subset of Rcv1v2 and apply PCA for dimensionality reduction, accounting for the feature dimensions in Table 4. We adopt a held-out evaluation protocol, allocating approximately 60%, 20% and 20% of samples to historical training, post-evolution training and testing, respectively.

Further clarifications on weaknesses are as follows:

(1) Theory. Our analysis characterizes component-wise predictive stability rather than end-to-end regret.

(2) Novelty. AFET does not claim individual computational components as novel. Its key contribution is shifting the focus from feature reconstruction to label-discriminative and relational evidence preservation under feature evolution. Fig. 1 motivates this distinction, while A9 demonstrates the complementary benefits of multi-label-specific mechanisms.

(3) Experimental design. Feature partitions are randomly generated. The existing S2 scenario uses only [S,A] with the same backbone. We will supplement imputation and complete-feature oracle comparisons, calibration diagnostics, and implementation details.

(4) One-pass protocol. For deep models, a single gradient update may be insufficient, particularly for small batches. We allow only 3 local updates per incoming batch to balance optimization and overfitting, without revisiting previous batches. We acknowledge the difference in update budgets and will clarify this relaxed one-pass protocol in the deep situation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.