acceptodds
Withdrawn from ICLR 2027

FedArena: A Unified Agent Harness for Federated Learning Poisoning Research

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Evaluating robust aggregation in federated learning (FL) requires attacks that adapt to the defense across clients and training rounds. LLM agents offer a way to explore such adaptations, but interpreting their results requires knowing what they observed, what they changed, and which changes actually affected training. We introduce FedArena, a unified harness for controlled, auditable agent-driven poisoning research. FedArena represents attack and defense operations as typed, composable objects, filters observations and interventions by an explicit threat model, and separates local defensive trials from real training rounds. It supports two agent modes, revising attack programs between complete FL runs (L1) and editing client messages between real rounds (L2), and compares both with scripted and human-designed attacks (L0). In an L1 case on CIFAR-10 against Divide-and-Conquer (DnC), a spectral outlier-filtering defense, the agent adds cross-round memory to an attack program; disabling the memory raises development accuracy by 1.98 pp. The frozen program, which also queries a loss-gradient oracle, ends with lower accuracy than the Min-Max attack on 4 of 5 seeds (54.5% vs. 58.0%). In 35 GPT-5.5 episodes on MNIST under Krum, adding local defensive trials raises the rate at which Krum admits controlled updates from 67% to 89%, yet mean final accuracy drops stay at or below 1.34 pp, below a scripted fixed program (3.74 pp) and Min-Max (7.89 pp). Statistics and full-message views reduce token use by 43–45% without increasing damage. Admission and damage thus diverge, and agent-driven adaptive evaluation should report both.

Primary area:
datasets and benchmarks
Keywords:
federated learning, poisoning attacks, LLM agents, agent environment, benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YLUsOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

In this paper, the authors introduce FedArena, which is a testbed or harness for studying LLM-agent-driven poisoning attacks in FL. There are three granularities: L0, which is a researcher-configured experiment; L1, where the agent revises the attack programs after every FL run; and L2, where the agent edits client messages and chooses observation timing within a run. They run experiments on MNIST and CIFAR with various attack and defense configurations. Overall, I like the idea behind the paper, but I have some concerns and questions, which I will list below.

Critical strengths
  • An LLM-driven harness for adaptive evaluation of robust aggregation is a very worthwhile and timely direction, and I appreciate the paper's ambition.

  • The L0/L1/L2 taxonomy is a useful conceptual contribution since it cleanly separates who revises the strategy and when the feedback arrives.

  • Overall, the paper is clear and easy to follow.

Critical weaknesses
  • My main objection is that the evaluation in this paper does not meet the bar this kind of contribution requires.
  • The paper uses MNIST and CIFAR, which are very old and basic datasets. They are not federated datasets and are synthetically partitioned to be used in this paper. For a testbed whose entire premise is federated realism, some sort of naturally partitioned dataset such as FEMNIST or Shakespeare etc., should have been used. All experiments should be run on at least one genuinely federated dataset.
  • The L2 part of the paper is on IID MNIST only. Why is it only on one dataset, and the easiest one, and why is it only in the IID setting?
  • It is hard to believe the success of FedArena if L2 is only evaluated under the IID assumption. That claim can only be made if it gives good performance in the non-IID setting.
  • 10 clients with full partition are used. With MNIST and CIFAR, that is not a good setting. There should be at least 100 clients to see the effect of federated clients properly. You can try both settings. 100 total and 100 per round, and 100 total with 10 per round.
  • 20 rounds of FL is very short to make a claim about cross-round memory and history-aware defenses.
Questions for authors
  • How does the L2 experiment fare under highly heterogeneous settings? This is my main concern with this paper.
  • Can FedArena support a naturally partitioned Federated dataset such as FEMNIST?
  • What would happen with a larger client pool and partial participation per round?
Reviewer_TT5EOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces FedArena, a unified harness for controlled, auditable agent-driven poisoning research. It supports two agent modes, revising attack programs between complete FL runs (L1) and editing client messages between real rounds (L2).

Critical strengths
  1. The research question is important and interesting.
  2. This paper suggests that higher admission does not yield greater damage, which is a useful observation for evaluating attacks.
Critical weaknesses
  1. The scope of the evaluation is limited. The main experiments cover only two datasets (CIFAR-10 and (MNIST)), two model architectures (CNN and LeNet), and two defense configurations (DnC and Krum). More experiments across multiple datasets, defense strategies, and non-IID settings are anticipated.
  2. The L1 and L2 experiments use different datasets and defenses, making it difficult to directly compare the two modes.
  3. The analysis of existing defenses is still inadequate. For example, in addition to DnC and Krum, there are many other state-of-the-art methods that have not been adequately discussed. A more comprehensive summary of the related work is strongly recommended.
Questions for authors

see weaknesses

Reviewer_fw81OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces FedArena, a unified harness for controlled, auditable agent-driven poisoning research, toward federated learning. It represents attacks and defense operations as typed, composable objects, enabling new methods plugging in. The extensive experiments were conducted for evaluates agents as poisoning attacks comparing with scripted fixed-program and human-designed attacks such as Min-Max.

Critical strengths
  1. This paper proposed the agent-driven poisoning research, which is interesting and timely, sit at the hot intersection of federated learning security and agent.

  2. The proposed methods are innovative and interesting. The similar idea can give insights to other fields in agents.

  3. The experiments are controlled and well-designed with sufficient ablation study and analysis.

Critical weaknesses
  1. The feedback and information-view experiments use only gpt-5.5, more architecture should be included.
  2. The definition of core metrics (admission, final drop, and AUC drop), formula of L1 memory mechanism, and the formalization of the search space should be placed in the appendix.
Questions for authors
  1. In 5.1, it is unclear for me why L1 and L2 are run on the same datasets and defenses (CIFAR-10/DnC vs. MNIST/Krum).

  2. It is unclear for me why gpt-5.5 is enough in the feedback and information-view experiments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.