The technical contribution appears somewhat incremental. Training an encoder through the critic loss while stopping gradients from the actor is already established in prior work [1,2], as is the use of self-predictive objective in the style of JEPA/SPR [3,4], or the exogenous Block MDP formalization [5]. The main novelty therefore seems to be the combination of these existing components and its extension to the federated setting.
However, the federated framing appears somewhat stronger than the empirical evidence warrants. The results suggest that federation contributes only modestly to the overall gains, as the advantage over the raw baseline also holds in the single-agent setting across all ten seeds. This leaves a gap in the paper's story: while the paper is framed around federated RL, a substantial part of the performance gains comes from the encoder and also carries over to the single-agent setting. It therefore remains unclear what the federated setting itself contributes to the broader insight.
Synthetic distractor setting. All distractors are appended i.i.d. Gaussian (or AR(1)) coordinates known a priori to be reward-irrelevant. It is unclear how well this setup reflects nuisance signals in real-world applications, which limits the generalizability of the findings. Including, for example, distractors that are partially correlated with the reward could strengthen the paper.
Auxiliary-objective comparisons are confounded by optimizer choice. Using different optimizers for different objectives makes it difficult to determine whether the observed differences are due to the objectives themselves or to the choice of optimizer.
Minor points that did not affect my overall recommendation: The paper contains several line breaks within inline equations, for example in Lines 215, 735, and 744.
Overall, these concerns currently outweigh the strengths of the paper. However, I'm open to revising my assessment based on how the authors address the concerns and questions raised in my review during the rebuttal phase.
To be fully transparent, the appendix is over 56 pages long, so I did not check every detail line by line. Instead, I spot-checked the key claims about the mechanism and the theoretical analysis.
Response to Reviewer 8NEP
Thank you for your time, and for flagging your confidence level. The summary comment above lists the changes in the revision, and our replies to Reviewers Dw1n and KzaE address their points. We would be glad to answer any questions.