- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies "temporal score incomparability" in continual local learning, assuming a Forward-Forward-style training as new classes arrive in a class-incremental stream. It proposes AT-LPC, a replay-based post-hoc calibration that subtracts a per-class tempered log-partition statistic from scores, with the temperature automatically selected. Experiments are run on CIFAR-10/100 and Tiny-ImageNet-200 in a one-pass class-incremental streaming. It shows accuracy gains over uncalibrated local experience replay.
Critical strengths
The paper studies continual learning in the specific case of forward-forward training. Even if it's not clear whether it's the way to go, it advances the field in an original direction
Critical weaknesses
I see two key critical weaknesses:
- Novelty and significance are not established. Classifier score incomparability is a well-known problem in continual learning, and the paper's claimed novelty rests on it being specific to the Forward-Forward/local-learning case but it never clarifies what is actually specific to this setting, nor why naively adapting existing classifier-correction methods (LUCIR, SDC, AANets, FOSTER, DATS, none of which are discussed) would be insufficient. This is consequential because, without this, we cannot determine whether the paper offers a genuine contribution or is applying a known technique to a narrow variant of an already well-studied problem. My ability to judge this is further limited by the overall clarity of the writing, which obscures the paper's positioning and makes a confident assessment difficult.
- The evidence needed to substantiate the central claim is weak, hidden, or missing. The one ablation that would justify the method's necessity that existing BiC/Weight-Alignment-style corrections barely help in this setting (Table 7) is relegated to the supplementary material with no seeds or error bars (unlike the five-seed main results) and no explanation for this behavior. Similarly, the one direct comparison to the closest prior work (SFFA) is single-seed, hidden in the appendix, and reaches near-chance accuracy without clear discussion. No classical regularization-based CL baselines (EWC, SI, GEM/A-GEM) or recent classifier-correction methods are tested either. This is consequential because the paper's core empirical claim that this specific correction is necessary and effective in this setting is not adequately supported by complete, well-explained, and properly powered evidence.
Questions for authors
I have numerous comments that are not all critical, but altogether they make the paper clearly not ready for publication
- Writing. Overall, the writing is unclear, and substantial work is needed to make the paper easily understandable. This may stem from heavy reliance on AI for writing. I have tried to give my best assessment of the paper, but that assessment is bounded by how much I could understand of the method and its positioning given the current state of the writing.
- Background (Sec. 3.1). The paper does not provide the necessary background to understand the fundamentals of Forward-Forward techniques. Describing the class-incremental stream setting explicitly would also help.
- Motivation for the setting. The paper targets a very niche setting: class-incremental + streaming + Forward-Forward. It fails to explain why this specific combination is particularly relevant.
- Figure 2 is not comprehensive:
- The axis captions are not clear to me.
- What does each point represent? At what point in training is it taken?
- What is the meaning of the different colors?
- I assume the colored lines below the subtitle have no meaning — if so, I would drop them.
- Figure 3 is not very informative:
- It lists several parts of the method without connecting them, so the overall pipeline is missing.
- Listing all the equations without explanation doesn't help; I'd recommend dropping most of the equations from the figure and instead focusing on explaining the overall pipeline and its motivation.
- I don't see the point of showing the three Gaussians in panel 4. What are we meant to see in the histogram in panel 5?
- The role of caching is not clear, and the text doesn't clarify this point either.
- Section 3.2. The motivation for this formulation is unclear. Is the formulation itself novel, or is the novelty in applying it to this task? What is the actual motivation here?
- Missing related work on classifier calibration. Classifier calibration is a well-studied problem in continual learning, and several
relevant works are missing from the discussion and potentially from the experiments:
- LUCIR — Hou, Pan, Loy, Wang & Lin, "Learning a Unified Classifier Incrementally via Rebalancing" (CVPR 2019)
- Semantic Drift Compensation (SDC) — Yu et al., CVPR 2020
- AANets — Liu, Schiele & Sun, "Adaptive Aggregation Networks for Class-Incremental Learning" (CVPR 2021)
- FOSTER — "Feature Boosting and Compression for Class-Incremental Learning" (ECCV 2022)
- DATS — "Distance-Aware Temperature Scaling for Calibrated Class-Incremental Learning" (2025)
- Figure 4: please include as a vector format.
- Experimental evaluation is weak for several reasons:
- The one baseline from the closest prior work (SFFA), which is cited, is tested only on a single seed and hidden in the appendix. It reaches near-chance accuracy, which questions the validity of this comparison, as it does not properly establish a real comparison to SFFA's own mechanism.
- The ablation that motivates the proposed method's necessity — namely that existing BiC/Weight-Alignment corrections barely help in this setting (Table 7) — is also hidden in the supplementary material, with no seeds/error bars, unlike the five-seed main results. There is no detailed explanation for this unexpected behavior. This point and the previous one are not minor details: understanding why these baselines underperform is exactly what would justify and motivate the development of the proposed method. This should be clearly explained in the main paper.
- Even granting that the setting is different (Forward-Forward-specific), no classical regularization-based CL baselines (EWC, SI, GEM/A-GEM) are tested, despite these being cheap to adapt given that each local block already computes its own gradient. If there is a reason this is not straightforward, the authors should include that discussion in the paper.
- No comparison is made to the closest recent classifier-correction methods (LUCIR, SDC, FOSTER, and especially DATS, which is also a temperature-based calibration method for CIL).
- Evaluation is performed only on small and not very diverse datasets, which seems limited.
Suggestions for future work :
- Add background on Forward-Forward fundamentals and the class-incremental stream setting in Section 3.1.
- Motivate why the class-incremental + streaming + Forward-Forward combination is a practically relevant setting.
- Discuss classical CL classifier-correction methods (LUCIR, SDC, AANets, FOSTER, DATS) in related work.
- Provide Figure 4 in vector format.
- Broaden the experimental evaluation beyond CIFAR and Tiny-ImageNet.