acceptodds
Under review as a conference paper at ICLR 2027

JointLoom: Compact-State GPU Simulation and Streaming Imitation Learning for Multi-Agent Pathfinding

Abstract

Multi-agent path finding (MAPF) requires multiple agents to reach their goals without conflicts. While search-based methods have long dominated MAPF, learned models are increasingly used to predict actions or guide search. Their execution and training require evolving joint states to continuously supply structured model inputs, creating a new computational bottleneck between MAPF state evolution and neural computation. To address this challenge, we propose , a compact-state GPU reconstruction framework for efficient learning-based MAPF. 1) introduces a reconstructible-state contract that preserves complete joint configurations and consistent input semantics, 2) performs on-demand GPU reconstruction with dependency-aware reuse to reduce redundant input construction and transfer, and 3) asynchronously streams compact states to reduce learner waiting during expert-supervised training. Extensive experiments with graph-based MAGAT and token-based MAPF-GPT validate faithful reconstruction across distinct learned representations, with up to faster policy execution and faster online expert-supervised training. Source code is available at https://anonymous.4open.science/r/JointooM.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Machine Learning Systems, Multi-Agent Pathfinding, GPU Acceleration, State Representation, Imitation Learning, Training Systems

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Rsn5OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

JointLoom accelerates learning-based MAPF by replacing producer-side policy inputs with complete compact joint states plus policy context. GPU builders reconstruct backend-specific MAGAT graphs or MAPF-GPT tokens on demand, using explicit cache-dependency and storage-lifetime rules. The same interface supports GPU-resident execution and streaming LaCAM imitation learning. Experiments report semantic checks, timing, mechanism ablations, reduced learner waiting, and matched-budget training progress.

Critical strengths
  1. The compact-state reconstruction interface is reusable across execution and expert-supervised training, and is validated with both graph-based MAGAT and token-based MAPF-GPT.

  2. The evaluation is unusually rigorous for a systems paper: 11,344 semantic comparisons, input/logit/action checks, Amdahl bounds, and separate reconstruction versus buffering controls.

Critical weaknesses
  1. No end-to-end MAPF solution-quality results. The paper reports latency, throughput, and action accuracy, but not success rate, sum of costs, makespan, conflicts, or standard MAPF benchmark outcomes. Thus faster training and execution are not connected to better task performance.

  2. Main speedups do not isolate the proposed interface. The 3.79x complete-decision result compares hand-written CUDA with the authors' JAX/XLA–PyTorch implementation, changing kernels, framework composition, synchronization, and builders simultaneously. The stronger cached-prefetch CPU baseline reduces the training gain to about 2x, while the headline 40.8–41.2x update improvement is against a synchronous host-expanded baseline.

  3. Generality and external validity are narrow. Only MAGAT and MAPF-GPT builders are implemented; recurrent-state and guided-search interfaces remain future work. Evidence uses one workstation/GPU, mostly fixed Bernoulli maps, three MAGAT seeds, and one online MAPF-GPT seed. The reported N A^2 graph backing and goal caches, rather than compact transport, limit the largest tested scale.

Questions for authors
  1. Can you report standard MAPF solution-quality metrics for the trained checkpoints, or explicitly justify why task-level outcomes are out of scope?

  2. Can you add a control that separates compact-state delivery from CUDA/framework implementation advantages, such as an optimized GPU-resident JAX/PyTorch builder?

  3. Why is the matched-budget headline based on the synchronous CPU baseline rather than cached prefetch? Please include accuracy versus update count and uncertainty for the stronger baseline.

1 comment
AuthorsOpenReview ↗

Task outcomes, optimized controls, and input-interface coverage

Thank you for your careful review and for highlighting the connection between training acceleration and MAPF task performance.

W1/Q1. Task outcomes and trained checkpoints

The original fixed-budget study measures throughput and frozen-trajectory expert-action accuracy in a controlled online imitation workload, separate from released MAGAT+ large-scale pretraining and scene adaptation. S4b reports those checkpoints' 256-step task outcomes.

To address task quality, we added complete 52-epoch scene adaptation from released pretrained MAGAT+ 195.pt with live LaCAM correction (S8). Validation-selected JointLoom/CPU models solve 144/150 and 145/150 held-out tasks, from 111/150 at initialization. JointLoom's successful-task mean SoC/makespan is 4,595.42/132.81; S8 also gives paired costs on 139 common successes. The CPU/JointLoom runs take 4h01m55s/2h27m28s (1.64× wall-clock ratio), with actual updates and corrections reported. JointLoom thus supports complete online adaptation with improved MAPF task success.

Under the same official evaluator, window-128 fine-tuning solves 97/128 128-agent warehouse tasks at the fixed-final checkpoint and 118/128 at the validation-selected checkpoint, from 47/128 initially. On 36 common initial/final successes, SoC falls 31,840 to 13,576 and makespan 415.6 to 235.8 (S10). This links windowed compact-record training to improved task success and lower path costs from initialization.

W2/Q2. Implementation, representation and buffering

We appreciate your request to separate delivery and implementation effects. The fully warmed CAMAR-framework discrete MAPF adaptation matches state, checkpoint, precision and goal caches. At 256 env × 256 agents, CAMAR/JointLoom ready-input time is 2.8735/0.3945 ms (7.28×); complete decisions are 156.13/152.86 ms (1.02×). All JIT variants, including post-DLPack, are warmed; timing ends at completed successor state (S2). This confirms a ready-input advantage against a GPU-resident reference.

With the same CUDA builder, learner and records, synchronous/ahead-one-batch prebuilding gives 0.995×/1.021× on-demand throughput; expanded-view storage is about 198× compact storage. Fixed reconstruction with ring depth 1→128 gives 2.96× throughput and wait 67.3% to 6.9% (S7). These controls establish compact-storage savings and throughput gains from buffering.

With common Pogema transitions, queries and learner, three 1,024-update pairs match actions, labels, updates, final model and Adam states exactly. JointLoom achieves 2.09× online / 2.07× full-process speedup (S9). This establishes shorter execution with exact learning-state preservation.

W2/Q3. Cached CPU and learning curves

The main streaming comparison uses tuned cached CPU: 1.98× (1.92-2.12) at eight experts, 2.08× (1.95-2.21) at sixteen, and 2.44× under matched eight-core allocation (S1). Synchronous host expansion is a mechanism control.

Figure S1 shows each seed against updates and wall time. At the shared cap, JointLoom/cached CPU final medians are 78.57%/76.49%. Near 20k updates they are 75.81% (75.25-76.27%) / 76.74% (53.32-76.95%). Curves use H2/Q4 logging; throughput uses H8/Q16. Thus, the time-budget comparison pairs faster training with higher median expert-action accuracy.

W3. Input families, coverage and capacity

Graph, token and CostTracer-architecture CNN inputs cover execution, IL and PPO. Native asynchronous SF and JointLoom complete 0.86435B/1.20416B learner samples in one hour (1.393×); final policies complete 8,730/8,767 goals on unseen mazes (S5). This demonstrates higher one-hour PPO throughput with effective lifelong policies.

MAGAT+ uses 1,000 23x23 warehouse tasks with 32-80 agents; S10 also tests 128 agents on a 32x32 warehouse. PPO trains on three mazes/one warehouse and tests two unseen mazes. S1/S3/S6/S9 use three performance/control seeds; S5/S8/S10 each use one paired training seed.

All hardware timings use RTX 5080 with stated CPU budgets. At 32 env × 128 agents, SF/JointLoom throughput is 257,277+/-909 / 323,857+/-414 samples/s (mean +/- SD over three seeds), using 4.14/1.00 core-equivalents under eight-core affinity (S6). At 256 env × 256 agents, 16-env microbatches reduce peak allocation 8.532 to 2.073 GiB, retaining 1.630 GiB persistent storage. Goal caches, O(N A^2) graph backing and model workspace are accounted for separately. Microbatching improves capacity by reducing model workspace at fixed persistent allocation.

The interface taxonomy specifies recurrent hidden-state update/reset and planner-owned frontier/priorities separately from evaluated builders.

S tables and Figures S1/S2: revised-PDF Appendix A. Records/configurations: JointLoom_Rebuttal_Data.zip and extracted files under rebuttal/data/ at the original anonymous link. Code/data: https://anonymous.4open.science/r/JointooM

Reviewer_DvYUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses the speed training performance problems of learning-based multi-agent path finding (MAPF) pipelines. It proposes JointLoom approach, which speed up observation generation provided by an expert (e.g. LaCAM) to learned policies, and rebuilds the inputs (MAGAT graphs, MAPF-GPT tokens) on the GPU. The paper reports speedups for both policy inference and supervised training.

Critical strengths
  • Transferring learning-based MAPF pipelines fully to GPU is a promising direction: fast shared input construction (e.g., tokenization, graph building) could serve many learned MAPF methods and, given MAPF's existing benchmarks and expert datasets, have broader impact, e.g., as a strong testbed for large-scale MARL, including offline-to-online MARL.
Critical weaknesses

W1: The paper primarily closes an engineering gap in learning-based MAPF by combining GPU input reconstruction, caching, and buffered data delivery. Although practically useful, the contribution appears limited and narrowly scoped.

W2: The training study is a toy example: MAGAT+ is trained on four 65x65 Maze maps for about 50 minutes, and progress is measured only as agreement with LaCAM actions on four fixed validation trajectories. The paper does not measure how its changed training protocol (online streaming, in-order consumption, few maps) affects the policy's actual MAPF performance, e.g., success rate or sum of costs.

W3: The main experiment trains on four fixed maps with LaCAM expert trajectories that do not depend on the learned policy, so the data can be generated ahead of time. A standard offline pipeline - precompute observations once, then stream them with a prefetching loader, as MAPF-GPT's released pipeline does - would remove the per-step reconstruction cost entirely.

W4: The main training baseline (cached CPU prefetch) is the authors' own implementation, evaluated in a single configuration (8 CPU builders, 16 prefetch slots), and its code is not in the linked repository.

W5: Proposed approach stream data (observations) strictly in publication order, with no shuffling. In the matched-budget study, each update is one snapshot from each of four episodes, and the next update is the next timestep of the same episodes, so consecutive batches are highly correlated. The paper does not test whether this affects the learned policy, or whether shuffled consumption would keep the speedup.

Questions for authors

Q1: How does training on the ordered, unshuffled stream affect the final policy's MAPF performance (e.g., success rate, sum of costs) compared with training on the same expert data with shuffled minibatches?

Q2: Can shuffling (e.g., a shuffle buffer over compact records) be added while keeping the speedup, given that goal-cache reuse relies on consecutive stages sharing goals?

Q3: How proposed approach can be adapted to lifelong MAPF, where agents continuously receive new goals?

1 comment
AuthorsOpenReview ↗

Same-data shuffling, task quality, and online policy feedback

Thank you for your careful review and your request to compare sampling orders on the same data.

W5/Q1. Same-data sampling and task quality

JointLoom fine-tunes released MAGAT+ 195.pt with identical initialization, corpus, correction-release epochs and 29,124 updates over 52 epochs across ordered/window-128/global sampling. All use seed 42, the official evaluator, common task/action seeds and a 512-step horizon.

Checkpoint/orderHeld-out tasks128-agent warehouse
Initial 195.pt111/15047/128
Final ordered134/150100/128
Final window-128136/15097/128
Final global shuffle135/15085/128

On 123/79 common successes, ordered/window mean SoC is 4,196/4,188 and 13,989/13,706. S10 gives makespans, paired costs and validation-selected checkpoints. All orders improve task success from the common initialization, demonstrating support for shuffled learning.

W5/Q2. Shuffling with cache reuse

Complete configurations are shuffled before publication, with goal/topology-dependent cache refresh. Including window fill and refresh, window-128 retains 2.474× online throughput over cached CPU prefetch; randomly selecting four ready sources among sixteen gives 2.254× (three-seed paired medians, S1). This demonstrates compatibility between sampling freedom, cache reuse and online speedup.

W2. Complete policy-feedback learning

Thank you for requesting task-level evaluation. The original fixed-budget study measures online-training speed and frozen-trajectory expert-action accuracy, using a controlled streaming-imitation workload separate from the released MAGAT+ pretraining and scene-adaptation pipeline. We added scene adaptation from pretrained 195.pt with live LaCAM correction (S8).

CPU online/JointLoom complete 52 epochs in 4h01m55s and 2h27m28s (1.64× wall-clock ratio). Validation-selected models solve 145/150 and 144/150 held-out tasks; JointLoom's successful-task mean SoC/makespan is 4,595.42/132.81. Both execute 6,500 rollouts with their own corrections; S8 lists actual updates/corrections. Three paired seeds match actions, labels, updates, final model and Adam states exactly with 2.09× online speedup (S9). Thus, JointLoom supports complete adaptation, and controlled execution preserves the exact learning result.

The initial corpus contains 1,000 tasks on a 23x23 warehouse with 32/48/64/80 agents (700/150/150 split). S10 tests 128 agents on a 32x32 warehouse with the same evaluator.

W1. Technical contribution

JointLoom's reconstruction contract preserves complete joint configurations and required context, such as token history, because observations and graph edges depend on joint state. GPU consumers rebuild views, reuse goal/topology-dependent data and protect storage through its final use. The shared graph/token/CNN interface serves execution, IL and PPO. S2/S7 isolate input construction, storage and buffering; S8/S9/S5 demonstrate reuse across online workflows.

W3. Pre-generation and online inputs

With fully materialized inputs and hidden delivery, static-data preparation precedes training. S3 tests RAM-resident raw states/labels: 4 env × 512 agents, batch 2,048, 4,096 updates, CPU H2/Q4 with cache/pinned lookahead. Three-seed paired gains are 5.19× ordered and 8.95× window-128; input construction and all learning work are timed. This establishes a reconstruction benefit when expert trajectories are pre-generated. Updated policies determine correction queries (S8/S9); PPO generates states and rewards online (S5).

W4. CPU tuning and implementation

Under eight-/sixteen-core budgets, pilots with H=2/4/8/16 and Q=2H select H8/Q16, the fastest in both. CachedHostBuilder/HostPrefetch are in src/mapf_cuda/training/host_inputs.py; experiments/runners/run_training_revision.py is the training entry point. The index identifies the raw-state CPU online adapter; the data package gives candidate scores/selection records. S3's H2/Q4 and Figure S1's logging cohort are distinct from H8/Q16 throughput. This supplies a tuned, inspectable CPU reference.

Q3. Lifelong MAPF

Arrival assigns a new reachable goal and refreshes goal-dependent context. At 256 env × 256 agents, lifelong inference is 6.18× faster than the original Pogema/MAGAT path with the same GPU policy. One-hour lifelong PPO processes 1.393× as many learner samples as native asynchronous SF; final policies complete 8,767/8,730 goals on unseen mazes (S1/S5). This verifies efficient lifelong inference and training with goal refresh inside the loop.

Tables and curves: revised-PDF Appendix A. Records/configurations: JointLoom_Rebuttal_Data.zip and extracted files under rebuttal/data/ at the original anonymous link. Code/data: https://anonymous.4open.science/r/JointooM

Reviewer_2X1DOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes JointLoom, a framework for learning-based MAPF. Producers pass only compact joint states to the GPU, which reconstructs policy inputs (MAGAT graphs or MAPF-GPT tokens) on demand with dependency-aware goal caches. An ordered pinned-memory ring buffer decouples CPU LaCAM experts from the learner. Experiments on a single GPU show faster policy execution and training than CPU and JAX baselines.

Critical strengths
  1. The paper shows that input construction, not model compute, can dominate learned-MAPF pipelines: about 50% of MAGAT’s training loop is spent waiting for inputs even with cached CPU prefetch. Treating input delivery as a first-class systems problem is a useful perspective for the community.

  2. Against cached CPU prefetch, MAGAT training speeds up by about 2× (2.44× under a fixed CPU budget), close to the Amdahl bound, while H2D payload per sample drops by about 200×.

Critical weaknesses
  1. The online-streaming setting is not motivated. LaCAM labels come from expert trajectories and do not depend on the policy, so they can be generated offline, as MAPF-GPT and the released MAGAT+ pipeline already do. In that regime the buffering contribution largely disappears. The paper needs a convincing case where online generation is required.

  2. Headline speedups rest on weak, self-built baselines. The JAX/PyTorch control shows large fixed overhead and unexplained host stalls, and the largest training gains are measured against a near-strawman host-expanded pipeline, so the reported speedups likely overstate the design’s actual advantage.

Questions for authors
  1. Compared with offline pre-generated LaCAM data plus an optimized GPU-side data loader, what benefit does JointLoom retain? In which concrete settings is online expert streaming necessary?

  2. Given the minimal gains for MAPF-GPT, does JointLoom remain useful as models grow, or is its benefit confined to lightweight policies such as MAGAT?

2 comments

There is no authors' rebuttal response. Therefore, I keep my rating for this paper.

AuthorsOpenReview ↗

Online feedback, optimized references, and input-cost savings

Thank you for your careful review and constructive questions. We address online data needs, optimized references and model-cost dependence below.

W1/Q1. Policy-dependent online data

In MAGAT+ adaptation, every four epochs, 500 current-policy rollouts identify unresolved joint states for LaCAM correction. Corrective trajectories train the next checkpoint, whose behavior determines subsequent queries. On-policy PPO likewise generates future states, observations and rewards from current actions. Thus, both workflows require interaction with the evolving policy to obtain subsequent training data.

Q1. Pre-generated trajectories and online adaptation

JointLoom accelerates simulation, input construction and delivery around neural computation. For a static corpus whose complete model inputs are preprocessed and delivery is fully hidden, that input work has already occurred before training; the residual loop is governed by shared neural computation.

S3 measures pre-generated raw expert trajectories, resident in RAM, with input construction inside training. Both paths consume the same records/order: 4 env × 512 agents, batch 2,048, 4,096 updates, same GPU learner and sixteen-core budget. Against cached CPU H2/Q4 with pinned H2D lookahead, three-seed paired gains are 5.19× ordered and 8.95× window-128. Timers include input construction, delivery, forward/backward and Adam; expert generation and disk loading precede timing. This establishes a training-speed benefit when expert trajectories are pre-generated and model inputs are constructed during training.

We added three online comparisons:

  • Identical learning: 16 env × 128 agents, three paired 1,024-update runs with common Pogema transitions and learner. Actions, query states, labels, updates, final parameters and Adam tensors match exactly. JointLoom completes the online loop 2.09× faster (S9).
  • Full policy feedback: CPU online inputs and JointLoom complete 52 epochs in 4h01m55s and 2h27m28s (1.64× wall-clock ratio). Both use RTX 5080, sixteen cores and four experts, executing 6,500 rollouts. Their 617/514 corrections and actual updates are reported alongside time. Validation-selected models solve 145/150 and 144/150, from 111/150 before adaptation (S8).
  • On-policy PPO: with common CNN initialization, CostTracer architecture, reward/PPO settings, GPU and eight-core budget, JointLoom processes 1.20416B versus 0.86435B learner samples in one hour against native asynchronous Sample Factory (1.393×). Startup and compilation are included (S5).

Together, these demonstrate efficient policy-dependent collection and learning; the identical-learning control establishes shorter execution with the same learned parameters.

W2. Tuned CPU and GPU references

We appreciate your emphasis on optimized references. The main streaming comparison uses cached CPU H8/Q16, selected from H=2/4/8/16 under each CPU budget, with cache reuse and pinned H2D lookahead. At 8/16 experts, exposed wait falls 50.4% to 2.5% / 52.8% to 2.4%. Per-sample wait w=f/q falls 97.5%/97.9% across three paired seeds. Measured 1.98×/2.08× gains approach the 2.02×/2.12× bounds for removing exposed wait at fixed baseline non-wait cost. Matched eight-core allocation gives 2.44× (S1). JointLoom therefore removes nearly all exposed acquisition wait against tuned CPU prefetch.

At 256 env × 256 agents, the CAMAR-framework discrete MAPF adaptation and JointLoom use matched state, checkpoint, precision and warm goal caches. Ready-input construction takes 2.8735/0.3945 ms: 7.28× and 86.27% time reduction. Complete decisions take 156.13/152.86 ms (1.02×). All runtime JIT variants, including post-DLPack variants, are warmed; the timer ends at completed successor state. Three process pairs and same-builder representation/buffer controls are in S2/S7. In T=C+R, shared policy work C dominates complete decisions; the input result establishes a construction advantage with both paths GPU-resident.

Q2. Compute-heavy policies

For MAPF-GPT, H2D falls 1,033 to 26 bytes/sample. With eight cores, cached CPU input wait is 5.09% of the loop: measured 1.043× approaches its 1.054× Amdahl bound. MAGAT's larger exposed input share gives the 1.98×/2.08× gains above. These results show that the benefit tracks removable input cost, as quantified by the Amdahl bounds.

Tables and curves: revised-PDF Appendix A. Records/configurations: JointLoom_Rebuttal_Data.zip and extracted files under rebuttal/data/ at the original anonymous link. Code/data: https://anonymous.4open.science/r/JointooM

Comments on the paper

1 comment

We thank the reviewers for their careful reading and constructive suggestions. The original fixed-budget study measures online-training efficiency using throughput and expert-action accuracy on frozen validation trajectories. We added complete adaptation from released pretrained MAGAT+, task-level evaluation, sampling controls and optimized references around JointLoom's compact-state reconstruction interface.

  • Complete 52-epoch MAGAT+ adaptation takes 4h01m55s with CPU online inputs and 2h27m28s with JointLoom (1.64× wall-clock ratio). Validation-selected models solve 145/150 and 144/150 held-out tasks, from 111/150 at initialization (S8).
  • Three paired 1,024-update online controls match actions, expert queries/labels, updates, final model parameters and Adam states exactly, with 2.09× online speedup (S9).
  • Same-corpus ordered/window/global fine-tuning reports success, paired SoC and makespan; window-128 solves 136/150 and 97/128 tasks, from 111/150 and 47/128 (S10).
  • Native asynchronous Sample Factory and JointLoom complete 0.86435B and 1.20416B learner samples in one hour (1.393×), with final task evaluation on unseen mazes (S5).
  • Tuned cached CPU prefetch is the main streaming reference (1.98×/2.08×). The fully warmed CAMAR adaptation gives 7.28× ready-input and 1.02× complete-decision gains (S1/S2).

Together, these results establish lower input overhead and faster identical-learning execution, and connect complete IL/RL training to measured task outcomes.

The revised PDF places all S tables and Figures S1/S2 in Appendix A. JointLoom_Rebuttal_Data.zip contains protocols, configurations, curves and per-task results; the ZIP and extracted files are under rebuttal/data/ at the original anonymous link. Code/data: https://anonymous.4open.science/r/JointooM

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.