acceptodds
Withdrawn from ICLR 2027

EMCoP: Efficient Multi-Task Cooperative Perception with Reinforcement Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Cooperative perception helps connected agents understand their surroundings by exchanging information from different viewpoints. Existing cooperative perception methods often optimize sharing for individual perception tasks. However, agents need to run multiple tasks simultaneously to support safety-critical decisions such as path planning and obstacle avoidance. For instance, combining a surrounding object’s position, its expected movement and road layout helps an agent plan a feasible path. A naive extension of state-of-the-art frameworks would be to exchange information for all tasks independently, imposing heavy strain on shared bandwidth. However, tasks can provide overlapping information: e.g., detection and segmentation can both indicate that a region is occupied. If either provides enough information to plan around it, sharing additional features from both may improve their separate predictions without improving the path. Motivated by this insight, we introduce EMCoP, which trains a single reinforcement-learning policy to consider all tasks together when deciding how much information each task should share in the current scene. By evaluating the decision space through its effect on the agent's planned path, the policy learns which combinations keep path errors and predicted collisions low while minimizing communication. EMCoP further reduces communication by identifying recurring situations where recently received information remains sufficient. We evaluate EMCoP in cooperative driving on V2X-Sim. Extensive experiments demonstrate that EMCoP is able to reduce communication volume by up to 85% relative to full sharing, where every agent broadcasts all task information without selection.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Cooperative perception, Multi-task communication, Reinforcement learning, Communication efficiency, Autonomous driving, Planning-aware learning, Rule-based skipping

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_k9wyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

EMCoP targets multi-task cooperative perception, where sharing features for every task separately wastes bandwidth because tasks overlap (e.g., detection and segmentation both signal occupancy). It trains one RL policy that decides how much each task shares per scene, rewarded by downstream path quality: low path error and few predicted collisions at minimal communication. It also skips transmissions when recently received info is still sufficient. On V2X-Sim, it cuts communication by up to 85% versus full sharing.

Critical strengths
  • Shifting the evaluation objective from single-task perception accuracy to downstream planning quality, and accordingly jointly allocating communication bandwidth across multiple tasks. The key observation that different tasks provide overlapping benefits to planning while incurring different communication costs is valuable and represents the most compelling and well-supported contribution of the paper.

  • Formulating communication scheduling as a constrained sequential decision-making problem. By accounting for the receiver's cached information and recognizing that current transmission decisions affect future communication needs, this formulation is more reasonable than frame-by-frame communication selection.

Critical weaknesses
  • The validation set contains only 10 scenarios and 850 steps. All results are reported from single runs, without multiple random seeds, variance estimates, or statistical significance tests.

  • The collision rates of 12.82% and 12.94% differ by roughly one trajectory. Nevertheless, the authors use this marginal difference to claim that the model has learned to filter out noisy features, which probably appears to be an overinterpretation of the results.

  • The normalized metric "gain retained" amplifies centimeter- or millimeter-level performance differences into seemingly impressive percentage values, potentially overstating their practical significance.

  • The planner itself is relatively weak, with a 3-second ADE of approximately 2.2 m and collision rates ranging from 13% to 22%. Moreover, the evaluation is conducted in an open-loop setting.

  • The paper lacks simple but strong non-RL baselines, such as selecting the top-k feature groups using the existing influence descriptors with a fixed transmission rate. Notably, the "Random rates + learned groups" variant already achieves 96.7% gain retained with only 49.6% communication cost. This suggests that much of the benefit comes from feature-group scoring, while the learned rate policy primarily reduces communication overhead. Its contribution to planning quality remains unclear.

Questions for authors

See the weakness.

Reviewer_8147OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EMCoP jointly selects communication rates and message groups across detection, segmentation, and motion prediction, followed by rules that suppress redundant updates. Offline V2X-Sim experiments report substantial task-message savings while retaining much of the planning benefit of full communication.

Critical strengths
  1. Joint allocation across heterogeneous perception tasks addresses a meaningful systems problem. Table 2 reports substantial communication savings while retaining much of the full-sharing planning benefit, supporting the practical relevance of the framework.
  2. The implementation accounts for delayed delivery, receiver caches, and task dependencies. A shared planner and evaluation protocol make the system-level comparisons more interpretable.
Critical weaknesses

The contribution of joint allocation is not sufficiently isolated. Section 4.1 and Appendix E compare different task-level selectors, while Table 3 ablates representations, rates, rewards, and rules. These results support the complete system, but do not establish the benefit of joint over independent task policies under matched conditions. This limits assessment of the central methodological contribution.

The robustness of the reported trade-off remains uncertain. Section 4.1 evaluates ten validation scenes, while Table 13 and Appendix C.9 describe validation-based checkpoint selection. Appendix B.3 also derives current speed from current and next recorded positions. These disclosed choices do not establish overfitting or method-specific bias, but leave uncertainty about performance outside model selection and with causal speed inputs.

Questions for authors
  1. Do you have a comparison between joint and independent task policies with matched descriptors, model capacity, training rewards, and communication budgets?
  2. Were the ten evaluation scenes also used to select the low- and high-communication checkpoints? If so, what evidence shows that the reported trade-off is robust to checkpoint selection?
  3. How sensitive are the results to replacing the speed convention in Appendix B.3 with measured current speed or a backward difference?

I would raise my rating if controlled comparisons and independent evaluation strengthen the evidence for joint allocation under causal inputs, and lower it if new evidence shows that the reported advantage substantially depends on evaluation choices rather than the proposed method.

Reviewer_4hD6OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors focus on learning cooperative perception features that support multiple downstream tasks and address the limitations of naive methods that exchange overlapping, redundant information for each task independently. The core component that achieves this objective is a common frozen trajectory planner that learns to evaluate the combined benefit through reinforcement learning. The action space includes communication rates and groups for all peers and tasks, and the authors propose applying PPO to learn the recurrent policy. Its reward combines serialized task-message bytes, trajectory displacement error, and a collision surrogate. A subsequent rule-fitting procedure uses paired omission trials to identify opportunities to reuse recently received features.

Critical strengths

Interesting work with a solid grounding in its motivation. I agree that redundant and overlapping features indeed block more efficient cooperative perception, and this makes a sensible objective for the proposed method. The method design in Section 3 is reasonable and supported by the results in Table 1 and Table 3.

Critical weaknesses

My major concern with this paper is its experimental design. Specifically, it fails to isolate the advantage of the proposed joint allocation under a clearly specified communication setup. Table 2 compares a centralized joint policy with adaptations that choose detection and segmentation independently, and leaves motion all-on for every baseline. Despite joint allocation being the central contribution, there is no matched independent-per-task PPO controller or simpler joint allocator trained against the same planning reward. Interestingly, the authors cite planning-oriented methods like CoDriving and Select2Drive, but I am curious why they don't evaluate and compare against these baselines. More crucially, the joint policy consumes current descriptors across senders, global uncertainty, receiver-wise cache validity/age, and pending deliveries, as described in Appendix C, while Appendix D.2 specifies task-payload accounting without explaining how those distributed controller inputs are collected and charged. In short, the current results establish a favorable trade-off for the particular replay controller, but leave both causal attribution and complete communication cost unresolved.

Meanwhile, I think the evaluation in this paper is overall too limited and selection-dependent to support the precision of the planning conclusions. Table 13 schedules repeated RL validation on the same ten scenes used for the reported 850-frame evaluation, and Appendix C.9 says the two operating points use selected checkpoints. No separate final test set, training-seed spread, or scene-level uncertainty is reported. In addition, the fixed planner is trained with losses explicitly encouraging gains from added tasks, and replay speed is computed using the current and next recorded ego positions. These choices are disclosed, but make the evidence specific to this offline planner and its input convention. It is like making a robot expert in an experimental facility does not guarantee its success in the field. For practical values, a held-out evaluation and uncertainty analysis are needed to establish how reliably the joint controller preserves planning quality. An open-loop overlap metric alone cannot establish downstream driving benefit.

Questions for authors

My main concerns are as mentioned above. I have a few follow-up questions on the method and experiments. Regarding the setup, where does the joint controller run, and how does it get each sender's current descriptors and each receiver's cache/pending state? Meanwhile, do all methods use next-recorded-position ego speed at evaluation, and can an existing replay use a causal speed input? Were Table 1's task-combination gains also checked without the task-set consistency/alignment losses that explicitly favor using additional task inputs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.