acceptodds
Under review as a conference paper at ICLR 2027

PRISM: Event-Driven Multi-Agent Resource Collection with Adaptive Preferences

Abstract

Multi-agent resource collection, exemplified by the Multi-agent Traveling Officer Problem (MTOP), requires mobile agents with limited capacity to capture spatially distributed resources whose availability evolves over time. Existing methods optimize only collected volume and rely on uniform-step formulations, which introduce uninformative no-op transitions and ignore the coupling among agents dispatched at the same time. Real-world dispatch, however, must also balance timeliness and fairness. We propose PRISM, which formulates MTOP as an event-driven asynchronous multi-agent semi-Markov decision process in which decisions are triggered only by agent availability. A state-aware context encoder models interactions among agents, resources, and dispatch history; a semi-autoregressive decoder generates joint assignments efficiently while preserving inter-assignment dependencies; and an adaptive preference head infers agent-specific weights over Resource Volume, Timeliness, and Fairness, letting agents specialize to their local opportunities and team needs. On four public datasets spanning crime, emergency medical services, and traffic, PRISM outperforms the strongest multi-objective baselines by 16.98%, 6.19%, and 4.81% on Volume, Timeliness, and Fairness, respectively. Code: https://anonymous.4open.science/r/PRISM-C5BA.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Spatio-temporal data mining, Stochastic resource collection, Traveling Officer Problem

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_YWK5OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces PRISM, an event-driven multi-agent framework for stochastic resource collection that formulates dispatch as an asynchronous multi-agent semi-Markov decision process (SMDP). The method couples a state-aware context encoder with a semi-autoregressive joint assignment decoder and an adaptive preference head that infers per-agent trade-offs over three objectives: Resource Volume, Timeliness, and Fairness. Across four public spatio-temporal graph datasets, PRISM reportedly outperforms both single-objective and multi-objective baselines on all three metrics.

Critical strengths
  1. I am not an expert in urban data mining, but the problem setting of asynchronous resource allocation seems relatively novel in the machine learning literature.
  2. Evaluations on four heterogeneous datasets with multiple runs and standard deviations reported; inclusion of multi-objective variants of strong baselines makes the comparisons more compelling than volume-only baselines.
  3. The code has been uploaded to an anonymous repository, enabling reproducibility.
Critical weaknesses
  1. First, I find it puzzling that the authors’ first contribution, reformulating MTOP as an SMDP, is relegated to the appendix. Moreover, this new problem formulation does not seem to be closely connected to the proposed method. In other words, while I can understand the need for this reformulation, I do not see a clear design rationale linking the new problem formulation to the proposed neural network components. Further, unlike previous approaches (both multi-agent problem settings), what are the distinctive features and interpretability of the proposed neural model? It reads like a stack of existing techniques.
  2. The positioning relative to neural combinatorial optimization is light; this could be deepened, e.g., see [1], [2], [3].
  3. “Arrival-time collisions” and feasibility masks are referenced but not formally specified (what constitutes a collision, how ties over arrival times/destinations are resolved, what mask logic is used). The role of “team-level objective weights” in control tokens is unclear given training uses fixed scalarization; whether these are ever varied or learned is not explicit.
  4. The mapping from EMS/PEMS08/METR-LA traffic signals to “resources to collect” is not fully justified; these datasets are forecasting benchmarks repurposed for dispatch, and the operational meaning of the three objectives (especially Timeliness) in these domains needs clearer grounding. Dataset-specific agent speeds are chosen ad hoc to reconcile heterogeneous distance matrices; sensitivity of results to these choices is not reported and may influence relative method behavior. Statistical significance (beyond mean±std) and robustness to reward scalarization weights are not provided; it is unclear how stable conclusions are if λs change.

References:
[1] Kim, Minsu, Junyoung Park, and Jinkyoo Park. "Sym-nco: Leveraging symmetricity for neural combinatorial optimization." Advances in Neural Information Processing Systems 35 (2022): 1936-1949.
[2] Luo, Fu, et al. "Neural combinatorial optimization with heavy decoder: Toward large scale generalization." Advances in Neural Information Processing Systems 36 (2023): 8845-8864.
[3] Drakulić, Darko, Sofia Michel, and Jean-Marc Andreoli. "GOAL: A generalist combinatorial optimization agent learner." International Conference on Learning Representations. Vol. 2025. 2025.

Questions for authors
  1. See the Weaknesses.
  2. Have you compared the constrained-first agent ordering with randomized or learned orderings? Does performance or stability change materially?
  3. Within an event, have you tried per-assignment advantages (e.g., reweighting by marginal contribution) instead of a single shared advantage to reduce potential intra-event credit bias?
3 comments

We thank the reviewer for their detailed review and thoughtful questions.

  1. Problem formulation. We will move the relevant problem formulation into the main text. We appreciate your bringing this matter to our attention.

  2. Core contributions. The following table summarizes the core contributions of this paper. The problem–design connections are described in the Introduction and the opening paragraphs of the Method subsections.

Problem addressedProposed neural network componentsConsidered in prior MTOP methods
Agent-specific objective tradeoffs and role specializationAdaptive Preference head, supported by auxiliary-supervised scoring heads and preference regularization×
Coupling among agent assignments within a dispatch eventSemi-autoregressive decoder with context updates×
Event-driven dispatch without virtual no-op paddingEvent-driven policy with a State-Aware Context Encoder×
Joint optimization of Volume, Timeliness, and FairnessObjective-specific scoring heads and preference-weighted score fusion×
  1. Neural combinatorial optimization. We will expand the discussion of neural combinatorial optimization and include relevant references. Thank you for your addition to this part.

  2. Collisions and feasibility.

  • An arrival-time collision means multiple agents reaching the same node simultaneously.
  • Assignments coinciding in both arrival time and destination are excluded by feasibility checks.
  • Section 3.3 specifies the mask logic: “A feasibility mask is then applied to remove unreachable nodes, nodes that cannot be reached within the remaining episode time, and choices that would cause arrival-time collisions with other agents.” These nodes are removed from the feasible candidate set. More specific implementation details are available in the released code.
  1. Team objective weights. Team objective weights remain fixed and serve both as inputs to the State-Aware Context Encoder and as coefficients for computing the final reward. Section 3.1 identifies them as control-token inputs, while Appendix C.6 explicitly states: “The preference weights are used both as team-level objective weights signals in the State-Aware Context Encoder and as coefficients for computing the multi-objective reward.”

  2. Dataset interpretation and Timeliness.

  • Traffic flow and speed can represent abstract resources for congestion prevention and speeding enforcement, respectively. Their use in forecasting does not preclude their operational interpretation in MTOP.
  • The Introduction defines Timeliness as rewarding arrival near a destination’s recent local peak, and Appendix C.2, Eq. (20), gives its mathematical definition. The other two objectives are equally explained in the same section. Section 4.2 further explains that optimizing Volume does not necessarily yield strong Timeliness: the objectives are related but not equivalent, motivating their separate treatment.
  1. Speed sensitivity. We have added speed sensitivity results, reported below. The results show that the model's performance on the three metrics increases steadily as speed increases, with no trade-off among the objectives. Therefore, the speed factor can hardly demonstrate the design advantage of the model itself.
Agent speed (NYPD)Volume ↑Fairness ↑Timeliness ↑
8 (default)4.69 ± 0.654.79 ± 0.1211.75 ± 0.74
165.12 ± 0.565.07 ± 0.1412.82 ± 0.64
245.84 ± 0.775.30 ± 0.0914.22 ± 0.91
  1. Scalarization weights. The effects of reward scalarization weights are already reported in Appendix C.6 and Table 7. The results show that varying the reward scalarization weights shifts the team's overall behavior while still maintaining coordinated multi-objective decision making.

  2. Agent ordering. We initially explored learned ordering, but its computational overhead was prohibitive, especially with larger fleets. Compared with random ordering, constrained-first ordering has a practical rationale: it prioritizes agents with fewer feasible choices. Ordering is not a core contribution of this work, so we did not investigate it further.

  3. Within-event credit assignment. We tested sequential per-assignment advantage decomposition using intermediate decoder contexts. It performed worse than shared event-level GAE while introducing greater complexity, as illustrated by the NYPD results below. We therefore retained the shared event-level advantage.

Variant (NYPD)Volume ↑Fairness ↑Timeliness ↑
with sequential per-assignment advantage decomposition3.98 ± 0.854.21 ± 0.4810.18 ± 0.62
PRISM (with shared event-level advantage)4.69 ± 0.654.79 ± 0.1211.75 ± 0.74

Thank you for the response. Part of my concerns have been addressed, and I have updated my evaluation accordingly.

Thank you for considering our responses and raising your score. We appreciate your time, constructive feedback, and engagement in the discussion.

Reviewer_pzvLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an event-driven multi-agent SMDP for MTOP. To tackle it, the authors propose PRISM with a state-aware context encoder, an adaptive preference head, and a semi-autoregressive decoder. Experiments on a series of testbeds demonstrate the efficiency of the proposed method.

Critical strengths
  1. The idea of formulating the task as an event-driven MDP is good.

  2. The network architecture is well designed for the task.

Critical weaknesses
  1. The training process does not seem to be standard PPO. The current training description is left in the Appendix. Please consider maintaining some necessary content about the adapted design in the main text.

  2. The compared baselines are somewhat old and lack works from the recent two years.

Questions for authors

The comparison seems to lack standard MARL solutions. I am not sure whether this is an omission or whether there is any challenge in adapting them to this task.

4 comments

We thank the reviewer for their positive assessment and helpful feedback.

  1. Training presentation. We will include part of the description of the training adaptations in the main text. Thank you for bringing this to our attention.

  2. Recent baselines. Our comparisons include the latest MTOP baseline, SATOP (2024). MTOP features event-driven decision-making, dynamically feasible actions, and coupled node-visit rewards. Non-TOP/MTOP methods therefore require appropriate modifications for task adaptation, which may raise fairness concerns. We thus included all recent TOP/MTOP methods as well as representative MARL and combinatorial optimization methods.

  3. Standard MARL baseline. MAPPO-MO, included in our comparisons, is a standard MARL solution. Its adaptation is described in Appendix C.3.

Thank you again for your positive assessment and helpful feedback. When you have a chance, we would appreciate any further comments on our responses. We are happy to clarify any remaining questions.

Thank you for your response. I maintain my original positive score.

Thank you for considering our responses. We appreciate your time and constructive feedback.

Reviewer_AQZzOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper models multi-agent resource collection on dynamic spatiotemporal graphs as an event-driven asynchronous semi-Markov decision process, jointly optimizing resource volume, timeliness, and fairness. PRISM combines heterogeneous state encoding, agent-specific adaptive objective preferences, and a semi-autoregressive decoder that scores in batches and allocates sequentially. Across four datasets, it achieves higher mean performance than the listed baselines.

Critical strengths

This paper addresses asynchronous actions, resource competition, and trade-offs among multiple objectives, a problem with clear practical value. The overall architecture is well developed. Event-driven modeling fits the task's temporal structure, while batch scoring and sequential allocation offer a reasonable way to balance computational efficiency with coordination among agents. Agent-specific preferences also provide flexibility in adapting to different local opportunities. The experiments cover four datasets and include multi-objective baselines using the same objectives, ablations, and efficiency analysis. Means and standard deviations over 10 independent runs provide solid empirical support for the full system under the metrics defined in the paper.

Critical weaknesses

With the sequential updates and accumulation in Eqs. (21) and (22), the inter-visit intervals telescope, leaving the score largely determined by each node's last visit time. This does not adequately capture how evenly service is delivered over time. Please add time-averaged node age or the maximum service interval to check whether the fairness conclusion still holds. The distinct benefits of the new mechanisms have not been sufficiently isolated. A structured joint-policy baseline with the same information, encoder, and budget is still missing, particularly a comparison of final policy quality and efficiency against a fully autoregressive approach. The method's incremental contribution therefore needs to be defined more clearly. The fixed-preference ablation uses only equal weights, which does not adequately represent a tuned fixed-weight approach. Score decomposition also receives auxiliary ranking supervision, while preference differentiation is encouraged by regularization. The respective gains from the architecture and the training objectives need to be disentangled further.

Questions for authors

Does the fixed-step ablation preserve equivalent real-time discounting, reward aggregation, and training budgets relative to the event-driven version? Please clarify how resources are depleted after collection and how capacity constraints enter the state representation and feasibility checks. For the 'without adaptive preferences' variant in Table 3, the macro-average relative decrease across the 12 dataset-metric cells is approximately 21.90%. Please check how the 19.1% reported in the text was calculated.

4 comments

We thank the reviewer for their careful evaluation and constructive suggestions.

  1. Fairness. We have added a comparison of time-averaged node age. PRISM achieves the lowest time-averaged node age on all four datasets. Temporal uniformity of repeated service is a further consideration beyond our objective. Our original fairness metric targets hotspot chasing by encouraging service to neglected nodes. It concerns whether nodes receive service.
MethodNYPD ↓EMS ↓PEMS08 ↓METR-LA ↓
LERK-GA-MO8.54 ± 0.229.82 ± 0.0810.45 ± 0.1010.79 ± 0.12
SDDQN-MO11.48 ± 0.2010.67 ± 0.1612.07 ± 0.0411.02 ± 0.10
PTR-MO10.97 ± 0.279.88 ± 0.1411.69 ± 0.0610.57 ± 0.13
DGAT-MO9.05 ± 0.1410.07 ± 0.0811.15 ± 0.0710.99 ± 0.09
SAIN-MO10.38 ± 0.3010.10 ± 0.1311.92 ± 0.1310.75 ± 0.15
SATOP-MO12.01 ± 0.1010.42 ± 0.1011.05 ± 0.1410.69 ± 0.11
MAPPO-MO11.13 ± 0.1511.31 ± 0.1710.51 ± 0.2110.43 ± 0.06
Hungarian-SC9.18 ± 0.129.16 ± 0.1211.43 ± 0.0610.19 ± 0.07
PRISM7.82 ± 0.068.93 ± 0.0510.42 ± 0.0710.13 ± 0.06
  1. Fully autoregressive comparison. We have added a performance comparison with the fully autoregressive decoder. The results show no significant difference in performance between the two designs. Table 5 compares the efficiency of the two designs. The results show that Semi-autoregressive significantly improves efficiency.
DatasetMetricAutoregressiveSemi-autoregressive
NYPDVolume4.72 ± 0.544.69 ± 0.65
NYPDFairness4.87 ± 0.234.79 ± 0.12
NYPDTimeliness11.71 ± 0.6511.75 ± 0.74
EMSVolume4.24 ± 0.174.47 ± 0.48
EMSFairness5.67 ± 0.145.69 ± 0.15
EMSTimeliness11.73 ± 0.4011.78 ± 0.55
PEMS08Volume9.71 ± 2.539.56 ± 3.66
PEMS08Fairness3.15 ± 0.193.01 ± 0.31
PEMS08Timeliness20.26 ± 0.2620.02 ± 0.99
METR-LAVolume17.29 ± 0.7617.02 ± 1.46
METR-LAFairness5.24 ± 0.305.48 ± 0.29
METR-LATimeliness19.16 ± 0.5019.06 ± 0.76
  1. Fixed preferences. We have added two additional non-uniform fixed-weight configurations to the w/o adaptive preference ablation. It can be seen that under different weight configurations the model's performance emphasizes different objectives, but all of them are significantly weaker than the performance of Adaptive preference.
Variant (NYPD)Fixed preference (VOL:FAIR:TIME)Volume ↑Fairness ↑Timeliness ↑
w/o adaptive preference1:1:13.77 ± 0.461.81 ± 0.0810.65 ± 0.52
w/o adaptive preference1:2:13.01 ± 0.274.54 ± 0.138.54 ± 0.29
w/o adaptive preference2:1:13.89 ± 0.393.62 ± 0.169.92 ± 0.50
Full PRISMAdaptive4.69 ± 0.654.79 ± 0.1211.75 ± 0.74
  1. Auxiliary ranking supervision and preference regularization. We have separately ablated auxiliary ranking supervision and preference regularization. Both ablations lead to substantial performance degradation. Unclear objective-specific scores and collapsed agent roles limit agents’ ability to prioritize objectives suited to the current conditions, reducing overall team returns.
Variant (NYPD)Volume ↑Fairness ↑Timeliness ↑
w/o auxiliary ranking supervision3.99 ± 0.564.21 ± 0.1610.18 ± 0.91
w/o preference regularization3.58 ± 0.434.11 ± 0.149.05 ± 0.65
PRISM4.69 ± 0.654.79 ± 0.1211.75 ± 0.74
  1. Fixed-step ablation. The fixed-step ablation preserves exactly the same equivalent real-time discounting, reward aggregation, and training budget as the full model. We will make this point more explicit in the text later.

  2. Resource depletion, capacity, and feasibility.

  • When an agent arrives at a node, it collects and exhausts all resources available at that node at that time.
  • Capacity constraints do not enter the state representation: “capacity” refers to the finite number of agents, while individual agents have no limit on the volume they can collect. We will add these clarifications to the paper.
  • Section 3.3 specifies all feasibility checks: “A feasibility mask is then applied to remove unreachable nodes, nodes that cannot be reached within the remaining episode time, and choices that would cause arrival-time collisions with other agents.”
  1. Numerical correction. We have corrected 19.1% to 21.9%. The calculation mistakenly used 4.41 instead of 2.41 for METR-LA Fairness in the "w/o adaptive preference" variant. We thank the reviewer for carefully identifying this calculation error.

Thank you again for your careful review. When you have a chance, we would appreciate your feedback on whether our responses and additional experiments address your concerns. We would be happy to clarify any remaining points.

Your responses addressed my concerns, and I've raised my score.

Thank you for considering our responses and additional experiments and for raising your score. We appreciate your careful review and constructive suggestions, which helped strengthen our evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.