- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
BISKY addresses multi-agent energy coordination when agents enter, leave, or change their behavior. Each agent trains its own PPO actor and critic while keeping raw trajectories local. A dynamic bipartite graph connects agents to behavior channels, which aggregate either belief-model adapter updates or policy gradients. Channels let newcomers reuse knowledge from agents with similar operating patterns. The paper evaluates peer-to-peer energy trading and EV charging, reporting stronger open-population performance than the evaluated baselines and ablations suggesting that dynamic routing and retained knowledge matter most under population changes.
Critical strengths
- Knowledge stored outside an agent's private model can remain useful when membership changes, while agents keep control of their own policies. This is a coherent approach to reconciling local learning with experience sharing. The belief variant's validation on separate complete episodes also provides a practical safeguard against harmful transfers. This combination makes the framework potentially useful beyond a fixed team.
- The evaluation contains meaningful evidence of coordination under changing participation. The two environments couple agents through different mechanisms: market interactions and electrical constraints. Population schedules are paired across methods, and the EV experiments disable centralized action projection, so an external optimizer does not automatically repair infeasible joint actions.
Critical weaknesses
- The paper does not establish when locally learned knowledge is compatible across agents. Behavioral similarity determines whom to share with, but does not by itself make model updates transferable. In the belief variant, adapters share an initialization while their underlying predictors remain local; the description does not establish whether those predictors remain aligned. The policy variant similarly merges gradients from local policies without specifying a common evaluation point. I cannot determine whether channels combine compatible updates or rely on local validation to reject mismatched ones. A precise account of parameter alignment and synchronization could resolve this concern, but it is needed to assess the proposed sharing mechanism.
- The results do not clearly connect the proposed mechanism to the reported advantage. Table 1 reports one BISKY entry without identifying which of its two sharing variants was evaluated. Table 2 provides variant-specific ablations, but leaves the environment and return normalization unspecified. The reader therefore cannot connect these promising component effects to the main cross-environment comparison. This matters because the gain over the strongest open baseline in energy trading is modest, making the ablation evidence particularly important.
Questions for authors
- What parameters are shared, frozen, or synchronized before merging? Please define the reference for the adapter updates in Equation (10) and the policy parameters at which the gradients in Equation (11) are computed.
- Which sharing variant produced each Table 1 result? Which environment and normalization formula produced Table 2, and how do its runs relate to the main comparison?
2 comments
Thank you for clarifying the parameter synchronization and experimental settings, which resolves my earlier questions. I still have a concern about the closed-to-open comparisons, since Table 1 uses different P2P sharing configurations and Table 2 uses different normalization references for the belief-sharing scores. Similar scores therefore do not, by themselves, show that the same configuration retains its performance when the population changes. This leaves the performance-retention claim insufficiently supported, so I maintain my original rating.
We thank the reviewer for the careful review and constructive feedback. We agree that behavioral similarity alone does not guarantee the compatibility of model updates, and we have clarified the parameter alignment and knowledge-sharing procedures in Section 3.
Belief variant (Eq. 10): Shared parameters and reference point.
In the belief-sharing variant, only the lightweight adapters are shared. Each agent's main forecasting model is trained on its own local data and is never synchronized with other agents. Thus, we do not assume that the underlying forecasting models are aligned.
All adapters share the same initialization, θinit, and each agent computes its update relative to this reference:
Δθi=θi−θinit.
The channel merges these updates, and each agent evaluates the merged update on its own held-out episodes before accepting it. Specifically, the merged update is applied to the agent's current adapter according to Eq. (10), and the agent retains it only if its local forecasting error does not increase; otherwise, αi=0.
The shared initialization provides a common parameter reference, but does not guarantee transferability across different local predictors. The local validation step addresses this issue by rejecting merged updates that worsen the receiving agent's forecasting performance. Table 2 supports its importance: removing validation reduces the normalized return to 60.1 in the open setting, below the no-sharing baseline of 68.3, compared with 89.5 for the full belief-sharing variant.
Policy variant (Eq. 11): Gradient evaluation and synchronization.
In the policy-sharing variant, all agents assigned to a channel use the same channel policy parameters θc, including the actor and critic, to select actions based on their own local observations.
Each member computes its PPO gradient at the common channel parameters θc, using its own locally collected trajectories. The channel merges these gradients according to Eq. (11) and updates θc according to Eq. (12). Thus, all gradients within a channel are evaluated at the same parameter point, rather than being directly merged from independently parameterized policies. Raw trajectories are never pooled.
When an agent joins or switches channels, it adopts the policy parameters θc of its newly assigned channel. Unlike a single globally shared policy, BISKY maintains separate channel policies for agents with similar operating behaviors, allowing knowledge to remain available as channel membership changes.
We have also corrected an inaccurate statement in Section 3. In the policy-sharing variant, agents select actions using their assigned channel policy; the previous description incorrectly stated that the channel policy does not select actions.
Clarification of Tables 1 and 2.
We have revised the captions of Tables 1 and 2 and Appendix C.2 to clarify the sharing variants, evaluation environments, and normalization.
In Table 1, P2P energy trading uses policy merging, with additional belief-adapter merging in the closed setting. EV charging uses belief-adapter merging.
Table 2 reports ablation results from P2P energy trading only, covering both belief-sharing and policy-sharing variants. These experiments use five independent random seeds from runs separate from those reported in Table 1.
The normalized scores use a random policy as the zero reference and SharedPPO, a shared actor-critic trained on pooled agent data, as the 100 reference, except for the open belief-sharing setting, which uses a different reference policy as specified in Appendix C.2. Scores are calculated from each method's return at the end of training and averaged across seeds. SharedPPO is included only as a reference because pooling raw observations violates our locality requirement.
Regarding the relatively small performance difference from POAM in open P2P trading, we have revised the Introduction and Section 4 to describe BISKY as achieving comparable performance.
Summary of update compatibility. Behavioral similarity determines which agents share knowledge, but it does not itself guarantee that their updates are compatible. In the belief variant, the common adapter initialization provides a shared reference, and local validation determines whether an agent accepts a merged update. In the policy variant, all agents within a channel compute gradients at the same channel parameters θc.
In both variants, raw observations and trajectories remain local, and no centralized critic is required.