- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper addresses open-world class-incremental learning where both classes and features evolve over time, with limited labels and no explicit task boundaries.
OFIGM maintains a bounded bipartite graph of instances and features, using message passing to maintain representations as new features arrive. It combines prototype-based classification, anchoring for historical knowledge retention, and a novelty detection mechanism that uses prototype distance, prediction uncertainty, and neighborhood consistency. New classes are created when accumulated candidates satisfy persistence and coherence criteria.
Section 5 outlines an extensive evaluation protocol covering SENC, OCIL, distribution shifts, and feature growth. However, Section 6 reports only preliminary experiments on Digits, Wine, and Iris against two simple baselines and a no-graph ablation, along with an external outbreak detection experiment.
Critical strengths
(1) Relevant problem formulation. Jointly handling emerging classes and evolving feature spaces is an interesting setting that deserves more attention. The bipartite instance-feature representation is a reasonable design choice because new features can be introduced without resizing the model's input layer. I also appreciate that Section 2.4 distinguishes this application from prior work on instance-feature graphs.
(2) Transparent reporting of negative results. Section 6.2 acknowledges that removing graph memory improves several results, including all reported metrics on Digits, while Section 6.3 reports weaker early-warning performance for the graph variant. Although these results weaken the contribution, reporting them explicitly is appreciated.
Critical weaknesses
(1) The experimental evaluation falls substantially short of what the paper proposes. Section 5 describes multiple evaluation suites, including SENC, OCIL, and feature-growth benchmarks, with several established baselines. However, none of the direct SENC competitors listed in Table 2 are evaluated in Section 6. Instead, Table 3 reports results on Digits, Wine, and Iris against a prototype baseline, windowed kNN, and a no-graph variant.
The statistical protocol is also inconsistent. Section 5.3 specifies ten seeds for tabular experiments, significance testing, and confidence intervals, whereas Section 6 uses only three seeds without these analyses. Section 5.1 proposes label budgets of {1%, 5%, 10%, 25%}, but Section 6.1 reports only 20%. These are not minor omissions; the experiments needed to support the main contributions have largely not been conducted.
(2) The reported results do not convincingly establish the benefit of graph memory. On Digits, the no-graph ablation outperforms the full model across the reported metrics. On Wine, the simple prototype baseline achieves better Macro-F1 (0.889 vs. 0.872), AUROC (0.962 vs. 0.946), and forgetting. The AUROC improvements highlighted against OFIGM-noFG are small relative to the reported variability over three seeds.
This raises a basic question about what the persistent feature graph actually contributes. The class-creation thresholds, prototype anchoring, pseudo-labeling, and graph loss are not independently ablated. More importantly, feature growth is a central contribution, yet there is no convincing experiment isolating performance under evolving features or comparing against relevant methods such as SAFC.
(3) The external evaluation and presentation further weaken the submission. Section 6.3 reports Macro-F1 of 0.474 ± 0.186 and early AUROC of 0.577 over only 11 outbreak episodes. The graph model also performs worse on the pre-confirmation early-warning metrics. These results are too limited to support claims of generalization to real-world outbreak detection.
There are also several issues with the manuscript itself. Equation (1) and multiple mathematical symbols in Sections 3–4 are incorrectly rendered. Algorithm 1 and Appendices A, D, and E are referenced but absent from the supplied manuscript. These omissions make it difficult to assess parts of the method and the external evaluation.
Questions for authors
(1) Could the authors provide results against at least some of the direct baselines listed in Table 2, such as SEEN, KNNENS, HANOL, or SAFC, on the SENC datasets? Without these comparisons, it is difficult to assess the contribution relative to existing methods.
(2) On Wine, the prototype baseline outperforms full OFIGM on several metrics. How do the authors reconcile this with the claim that persistent graph memory improves performance?
(3) Could the authors repeat the experiments with the ten seeds specified in Section 5.3 and report statistical significance? The AUROC differences against OFIGM-noFG are relatively small compared with the reported standard deviations.
(4) Where is the evaluation of the proposed feature-growth capability? A comparison against a fixed-schema variant or SAFC under controlled feature arrival would help establish this contribution.
(5) Could the authors ablate the persistence-based class-creation mechanism, particularly the minimum component size, coherence threshold, and age requirement? How do these parameters affect detection delay and false-novel rates?
(6) Section 5.1 specifies label budgets of {1%, 5%, 10%, 25%}, while Section 6.1 uses 20%. Which settings were actually evaluated, and are results for the proposed budgets available?
Minor Comments
(1) Several mathematical symbols in Sections 3–4 are incorrectly rendered, making some definitions difficult to follow.
(2) Algorithm 1 and Appendices A, D, and E are referenced but missing from the supplied manuscript.
(3) Section 6.2 contains an incomplete concluding sentence.
(4) The statement in Section 1 regarding the stability–plasticity trade-off needs grammatical revision.
(5) Section 5's proposed evaluation protocol should be clearly distinguished from experiments actually completed.