acceptodds
Under review as a conference paper at ICLR 2027

Towards Permutation Invariant Learning with High-Dimensional Particle Filters

Abstract

Sequential learning in deep models often suffers from challenges such as catastrophic forgetting and loss of plasticity. This effect is largely due to the permutation dependence of gradient-based algorithms, where the order of training data affects the learning outcome. In this work, we introduce a novel learning framework based on high-dimensional particle filters that yields approximately permutation-invariant results. We theoretically demonstrate that particle filters are approximately invariant to the sequential ordering of training minibatches or tasks, offering a principled solution to mitigate catastrophic forgetting and loss-of-plasticity. Next, we develop an efficient particle filter for optimizing high-dimensional models, combining the strengths of Bayesian methods with gradient-based optimization. Finally, through extensive experiments on continual supervised and reinforcement learning benchmarks, including SplitMNIST, SplitCIFAR100, and ProcGen, we empirically demonstrate that our method consistently improves performance, while reducing variance compared to standard baselines.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
permutation-invariant learning, continual learning, loss of plasticity, catastrophic forgetting, particle filter, high-dimensional

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WbBbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposed a particle filter-based method, WPF, to tackle the forgetting and loss of plasticity caused by order-dependence in continual learning and reinforcement learning. The paper shows theoretical proof that particle-filtering, under some assumptions, achieve approximately permutation invariant result. The developed method is more practical than conventional particle filtering as it combines with gradient descent and able to perform learning in high-dimension. Empirical result show WPF effectiveness in CL and Lifelong RL setting.

Critical strengths
  1. Diverse experiment settings, not only cover normal SplitMNIST/CIFAR100, it also covers Lifelong RL with ProcGen. To the best of my knowledge, not much paper cover both CL and Lifelong-RL together and good to see a method that works for both field.

  2. Technically solid and well motivated derivation that address the key weakness of particle filter in high-dimensional learning setting. In bayesian update, we sacrifice tractability for a solid belief, where we have a distribution instead of point estimate. Since the posterior is intractable, this method provide a theoretically motivated hybrid to make it computationally feasible.

Critical weaknesses
  1. Strong assumption made in Line 167-178 for Eq. 9, where C≈1C\approx1. However, this is unstable when ∀ C>1\forall \ C \gt 1 the bound is divergence (for infinite/large TT) and there is no ablation that C≈1C\approx 1 is reasonable for some small, finite TT (at least matching experiment setup TT). Also assumming ϵ\epsilon is small is not always true and lack empirical proof.

  2. One classic and easy way to reduce the forgetting due to order of training in both CL and RL is Experience Replay (ER) [1]. ER with small amount of replay sample is known to have very strong performance. It is unclear whether under same additional memory space, whether storing small amount of samples already outperform the need of have N copies of model parameters (N=100 basd on Line 363).

  3. Intuitively, under the assumptions made, I still do not see that how WPF helps in breaking the order-dependence in the learning algorithm. Each independently intialized weight are trained with same data sequence and updated with gradient descent and combined in a non-trivial, bayesian motivated weights. Thus, intuitively, I think what happenning is that with a larger N and diverse starting point, different trajectory will learn slightly differently and forget differently, each of them cover the blindspot of one another and the WPF essentially provide an mathematically grounded framework to combine all learned weights and more blindspots are covered. In my opinion, WPF is not practical in CL unless N is very small but I think it has better usefulness in other settings as it provide a grounded and effective way in combining knowledge.

  4. No ablation studies on key hyperparameter, especially the number of particles.

Minor:

, and . after each Eq

Nice to see great improvement, with great stability. But it is well known that increasing parameter space reduce forgetting a lot, no direct ablation isolating that factor [2][3].

Baselines compared are rather old (2017 or earlier), but practically, these methods are still extremely strong

Slight notation confusion of using xx for weight, but not a big deal

[1] Robins, Anthony. "Catastrophic forgetting, rehearsal and pseudorehearsal." Connection Science 7.2 (1995): 123-146.

[2] Dyer, Ethan, Aitor Lewkowycz, and Vinay Ramasesh. "Effect of scale on catastrophic forgetting in neural networks." (2022).

[3] Davari, MohammadReza, et al. "Probing representation forgetting in supervised and unsupervised continual learning." 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022.

Questions for authors
  1. What's the model utilized in the SplitMNIST/CIFAR100 experiment (apologies if I overlook, but I couldn't find it in the main paper nor the appendix)? Based on supplementary material code, it is two different custom defined CNN for SplitMNIST and CIFAR-100, I believe testing on standard ResNet or ViT will be a better practice.
  2. Is it possible to provide some preliminary, simple result to show that the assumptions made in Line 167-178 and 198-207 are reasonable?
  3. I am sorry for asking more experiments but will it be reasonable to compare against base model with same parameter / VRAM budget (e.g. Let N=10 in WPF, then let's say each x(i)x^{(i)} has 10 parameter, then maybe you should compare against a model train with SGD but with 10$\times$10 number of parameters, i.e. 1M to 10M param) to rule out the possibility that the reduce in forgetting and increase in plasticity is due to increase in parameter space.
  4. What is the VRAM consumption and computation speed of WPF compared with EWC (that requires roughly N=3)?
Reviewer_LMKaOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a gradient-based Weighted Particle Filter (WPF) for reducing catastrophic forgetting and loss of plasticity through approximately permutation-invariant learning. The method maintains multiple independently optimized models and dynamically adjusts their contributions using loss-based weighting. The authors establish theoretical bounds on permutation sensitivity and task-specific losses under stability and approximation assumptions, and demonstrate that WPF improves accuracy and reinforcement learning returns.

Critical strengths
  1. The paper provides an interesting perspective on continual learning by connecting catastrophic forgetting and loss of plasticity to task-order sensitivity.

  2. The proposed method demonstrates improved performance and reduced variance across task permutations on continual supervised learning and lifelong reinforcement learning benchmarks.

Critical weaknesses

The theoretical analysis is not rigorous, and assumptions are not well supported by evidence.

  1. This paper claims "particle filters offer a principled solution to mitigate catastrophic forgetting." However, permutation invariance is not equivalent to preventing catastrophic forgetting. The paper's theoretical analysis connects approximate permutation invariance to bounds on task-specific losses, but this connection relies on additional assumptions about (a) stability and (b) approximation error. It should provide experiments on (a) why C≈1C\approx 1 is reasonable; (2) is ϵ\epsilon sufficiently small error for practical neural networks.

  2. The proof of Theorem 3 appears to assume equal prior probabilities at the final particle locations (last equality of Eq. (45)), although the theorem only specifies this condition at initialization.

Questions for authors

In Line 363, the proposed method maintains 100 independently optimized models, introducing substantial training and inference overhead. What are the comparisons of training FLOPs, memory requirements, and inference costs? Do the observed performance improvements justify the additional resources?

Reviewer_yYp2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper concentrates on the effect of catastrophic forgetting and loss of plasticity in sequential learning, which is largely due to the permutation dependence of gradient-based algorithms, where the order of training data affects the learning outcome. It therefore introduces a novel learning framework based on high-dimensional particle filters that yields approximately permutation-invariant results. The main contributions can be summarized as

  1. Demonstrate that particle filters enable approximate permutation-invariant learning, where the algorithm’s output remains nearly the same regardless of the training data order.

  2. Introduce a simple, gradient-based particle filter specifically tailored for high-dimensional parameter spaces.

  3. Conduct experiments on continual and lifelong reinforcement learning benchmarks,

Critical strengths

1.This paper is well written and easy to follow.

2.Interesting ideas. This paper demonstrate that particle filters enable approximate permutation-invariant learning, where the algorithm’s output remains nearly the same regardless of the training data order.

Critical weaknesses

Main concern. In Theorem 1, the author provides an upper bound on CT−2C ^{T-2}, which is almost unacceptable. Even though the author discusses the case where C is approximately equal to 1 and believes that in this case, particle filters are approximately permutation invariant. However, at the exponential level, even a very small error between C and 1 can lead to a significant difference in the upper bound. Therefore, I believe that including exponential upper bounds in the results is unacceptable.

Questions for authors

Please address the concerns in Critical Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.