acceptodds
Under review as a conference paper at ICLR 2027

Decentralized and Lifelong-Adaptive Multi-Agent Collaborative Learning

Abstract

Decentralized and lifelong-adaptive multi-agent collaborative learning aims to enhance collaboration among multiple agents without a central server, with each agent solving varied tasks over time. To achieve efficient collaboration, agents should: i) autonomously identify beneficial collaborative relationships in a decentralized manner; and ii) adapt to dynamically changing task observations. In this paper, we propose , a decentralized multi-agent lifelong collaborative learning algorithm with dynamic collaboration graphs. To promote autonomous collaboration relationship learning, we propose a decentralized graph structure learning algorithm, eliminating the need for external priors. To facilitate adaptation to dynamic tasks, we design a memory unit to capture the agents' accumulated learning history and knowledge, while preserving finite storage consumption. To further augment the system's expressive capabilities and computational efficiency, we apply algorithm unrolling, leveraging the advantages of both mathematical optimization and neural networks. This allows the agents to `learn to collaborate' through the supervision of training tasks. Our theoretical analysis verifies that inter-agent collaboration is communication efficient under a small number of communication rounds. The experimental results verify its ability to facilitate the discovery of collaboration strategies and adaptation to dynamic learning scenarios, achieving a 98.80% reduction in MSE and a 188.87% improvement in classification accuracy. We expect our work can serve as a foundational technique to facilitate future works towards an intelligent, decentralized, and dynamic multi-agent system.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
graph learning, distributed training

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_bSw9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a decentralized lifelong-adaptive collaborative learning framework based on numerical optimization and algorithm unrolling. Experiments on image classification tasks and multi-robot mapping tasks show that the proposed method achieves superior performance compared with several collaborative learning approaches.

Critical strengths
  1. The proposed method is well‑motivated and addresses an important challenge for decentralized lifelong‑adaptive multi‑agent collaboration.
  2. The study appears to be theoretically sound.
Critical weaknesses
  1. Collaboration weights are determined based on pairwise model‑parameter similarity. However, parameter similarity may not indicate beneficial collaboration, especially in early training phases. Additionally, Section 2 is hard to follow. Many symbols are used before being formally defined when presenting the optimization problem, and the motivation for each term in the objective function is insufficiently explained. Readability would be improved if all notations were defined first and the rationale of each component articulated prior to introducing the full optimization formulation.

  2. The baselines are relatively outdated, and the evaluation tasks seems to be simple with small scale and little agents. More challenging scenarios, such as larger-scale datasets, systems with more agents or unseen task, would better demonstrate the method's effectiveness in complex collaborative settings. Moreover, the method lacks evaluation on communication efficiency.

Questions for authors
  1. What types of tasks is the parameter-similarity-based partner selection mainly suited for?
  2. Could you provide results or analysis comparing against more recent approaches?
  3. How does DeLAMA improve communication efficiency compared with existing methods?
Reviewer_YBs7OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors present a strategy for decentralized collaborative learning when agents have potentially varying objectives or data distributions. The proposed framework includes three phases, (1) local and independent learning, (2) using the models from stage (1) to find a collaboration graph, and (3) collaborative refinement of the local models.

To this end, they formulate an objective jointly over the model parameters Θ\Theta and task relationship graph WW, which includes a Laplacian smoothness term along with constraints ensuring the resulting adjacency matrix is valid (e.g., non-negativity, stochasticness, etc). This smoothness formulation for distributed multi-task learning is standard and can be found as early Neurips 2017 in the paper "Federated Multi-Task Learning" by Smith et al, where Mocha approaches the problem with primal-dual techniques.

Rather than alternate optimization over local models and the task graph, which would presumably lead to some global guarantee, local models are learning and fixed first using a Taylor expansion of the local loss functions (Step 1). This is followed by a distributed graph learning step over the learned model using standard methods (Step 2). Finally (Step 3), agents further refine their local models under the Laplacian smoothness regularizer, which is again fairly standard. In essence, rather than solving the joint optimization problem (1) directly, the authors have decomposed it into sequential steps, where each individual step is quite straightforward. Of course, if this approach could be shown to carry some convergence guarantee on (1), that would be great, but from reading Theorems 1 and 2, I don't get that impression.

Theorem 1 makes an asymptotic claim on the accuracy of the Taylor expansion, but there is no discussion on this error for finite (albeit small α\alpha) and how this will propagate into subsequent steps of the algorithm. Similarly, Theorem 2 establishes convergence relative to the target solution in (7), but this is isolated to Step 3 of the algorithm, and does not establish any sort of broader optimality. As such, there is no guarantee on end-to-end learning performance.

Critical strengths

The topic of decentralized multitask learning is timely and important. The framework proposed by the authors is intuitive and empirically effective.

Critical weaknesses

The overarching objective function (1) is classical and well-studied, hence the only contribution could be in the algorithm of theoretical guarantees. The algorithm proposed by the authors is straightforward, hence the only remaining contribution would be in strong theoretical guarantees. The theoretical guarantees fall short of expectation, and concern individual steps in isolation. They do not actually quantify the performance of the overall framework. Comparison with recent state-of-the-art approaches for decentralized multi-task learning is weak. I particular, federated and decentralized approaches used as baselines are single-task/consensus-based approaches, rather than more appropriate multitask or personalized models.

Questions for authors

What performance guarantees can you establish for the end-to-end performance of your proposed framework? How do the learned models and graph compare to the optimal solutions to (1)? What advantages does the proposed approach have relative to standard methods for distributed personalized learning? Can you compare performance empirically to more appropriate benchmarks such as Mocha and more recent references?

Reviewer_5A4tOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper develops a decentralized lifelong-adaptive multi-agent learning algorithm called DeLAMA. DeLAMA studies LML settings where agents receive non-stationary data streams and must autonomously decide whom to collaborate with, without a central server and without storing past data. DeLAMA jointly optimizes agent parameters and a sparse, nonnegative collaboration graph. DeLAMA uses alternate convex search in three steps: local learning, collaborative relational inference. Theoretical results show that DeLAMA minimizes the approximation error bound and has linear convergence for the parameter update. Experiments on class-incremental image classification using MNIST and CIFAR-10 show DeLAMA outperforms federated, lifelong, and decentralized baselines.

Critical strengths
  1. DeLAMA does not treat the collaboration graph as given or as a byproduct of a heuristic and makes WW a decision variable in the same objective as parameters, combining accumulated per-agent losses. This produces a single objective and also makes the static-graph assumption of FedAvg and related methods an explicit special case.

  2. The memory is a running average over all past tasks, not a replay buffer, so its size is fixed and independent of the number of tasks. This directly satisfies the constraint that storing past data is not feasible.

3.The performance margins are large across three different task types and ablative experiments confirm the claim that lifelong adaptation compounds over time. The regression results also show that improvements grow as timestamps increase.

  1. Most papers assume that collaboration strength should follow parameter similarity without checking it. DeLAMA runs a web-based regression study in which human participants draw regression lines and rate collaborators, and then compares human ratings with log parameter distance. The finding that humans struggle to identify useful collaborators under low data quality is an interesting observation and suggests the parameter-distance heuristic is not the whole story.

Weaknesses:

  1. The federated-learning baselines are standard approaches that were not originally designed for class-incremental learning. Comparing them directly against DeLAMA may not be entirely fair.

  2. The theoretical results (Theorems 1 and 2) assume linear models and convex losses. The actual implementation uses nonlinear models and an unrolled optimization procedure. The authors need to clarify which theoretical guarantees remain applicable.

  3. The paper motivates unrolling through reduced communication overhead and emphasizes decentralization and efficiency, but does not provide measurements of communication volume, communication rounds, wall-clock time, or memory footprint.

  4. The evaluation is limited to MNIST and CIFAR-10 with six agents and two five-class groups. The class-incremental protocol is not the standard Split CIFAR-10 or Split CIFAR-100 setup. There are no larger datasets, Transformer backbones, or large-model experiments to establish scalability. The regression and mapping experiments are also relatively small-scale or synthetic.

  5. The comparison against existing continual-learning methods is incomplete. Important methods such as DER, FOSTER, and prompt-based approaches are missing, despite the discussion of TAMiL in related work.

  6. The paper motivates decentralization through the absence of a central server but does not address privacy. Sharing model parameters does not inherently protect against gradient or parameter inversion attacks. No privacy mechanism, threat model, or leakage analysis is provided.

Questions:

  1. Ask whether the federated and lifelong-learning baselines were fairly tuned and properly adapted, including the hyperparameter search and the configuration of the FT variants.

  2. Ask the authors to resolve the discrepancies between Tables 2 and 5 and Table 1, specifically the MNIST and CIFAR-10 DeLAMA results, and clarify the actual GMSE comparisons against GLasso, L2G-PDS, and Oracle.

  3. Ask which theoretical guarantees survive the transition from the alternating optimization procedure to the unrolled network and nonlinear backbone, and whether the theorems apply only to initialization or to the actual implementation.

  4. Ask for communication and computational cost measurements, including communication rounds, bytes exchanged, wall-clock time, and Hessian computation cost as model size grows.

  5. Ask for the number of participants and trials in the human study and the confidence intervals and correlation statistics supporting Table 6 and Figure 8.

Please:

Expand these notes into a detailed review after reading the paper carefully and incorporate relevant details that grounds the review into the paper text. Organize the weaknesses and questions logically and avoid language that is not supported by the paper.

Critical weaknesses
  1. The federated baselines are standard methods not designed for class-incremental learning, yet they are compared directly against a framework built for it.

  2. Theorems 1 and 2 are proved for linear models with convex loss, and they concern the iterative optimization procedure. Because these are not valid in practice, the method does not inherits the convergence guarantee and moreover the unrolled network is trained by a different objective (expected task loss over sampled task sequences). It is OK to combine optimization with unrolling, but the theoretical section should say which claims survive and which do not.

  3. The paper's motivation for unrolling is explicitly the reduction of communication overhead and the paper emphasizes decentralization and efficiency. Yet there is no measurement of communication volume, number of rounds, wall-clock time, or memory footprint for DeLAMA versus baselines.

  4. The evaluation uses MNIST and CIFAR-10 with six agents and two five-class groups, a class-incremental scheme that is not the standard Split CIFAR-10 or Split CIFAR-100 protocol, making direct comparison with the continual-learning literature difficult. No ImageNet, no TinyImageNet, no Transformer backbone, and no large-model experiment are included, so scalability of the Hessian-based memory and the unrolled network is untested. The regression and mapping tasks add breadth but are synthetic or small-scale.

  5. The comparison set is outdated for continual learning: DER, FOSTER, and prompt-based methods are absent, even though TAMiL is discussed in related work.

  6. The paper motivates decentralization by the absence of a central server, but sharing model parameters in the clear does not provide privacy, and gradient- or parameter-inversion attacks are well documented in the federated learning literature. No privacy mechanism, threat model, or leakage analysis is provided.

Questions for authors
  1. Are the federated and lifelong baselines fairly tuned and properly adapted? Was each baseline given a hyperparameter search comparable to DeLAMA's learned hyperparameters, and were the "-FT" variants configured to make the strongest possible incremental-learning baseline? A matched-tuning comparison would clarify whether the gains come from the framework or from unequal effort.

  2. Can the Table 2 and Table 5 discrepancies be resolved? The MNIST DeLAMA row matches the NS row and contradicts Table 1's own accuracy numbers, while the CIFAR-10 DeLAMA row matches Oracle exactly. Which rows are correct, and what is the actual GMSE of DeLAMA against GLasso, L2G-PDS, and Oracle?

  3. Which theoretical guarantees survive unrolling and the nonlinear backbone? Theorems 1 and 2 assume linear models, convex losses, and the full alternate-search procedure. Does the single-iteration unrolled network retain any convergence or approximation property, or are the theorems only relevant to the initialization step? The paper should state explicitly which claims apply to which object.

  4. What is the actual communication and computation cost, and is one iteration enough? How many communication rounds, bytes, and wall-clock seconds does DeLAMA use relative to baselines? Is the Hessian computed exactly and at what cost as model size grows?

  5. What does the human study actually establish, and what are its statistics? How many participants and trials were run, and what are the confidence intervals on Table 6 and the correlation in Figure 8?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.