acceptodds
Under review as a conference paper at ICLR 2027

Interaction-Symmetric Decoding for Closed-Loop Trajectory Planning

Abstract

Modeling multi-modal driving behaviors under dense agent interactions remains a fundamental challenge for autonomous closed-loop trajectory planning, and recent generative planners such as Diffusion Planner and Flow Planner have emerged as the dominant paradigm. However, these planners inherit a one-sided scene conditioning pattern from their transformer-based decoders, in which trajectory tokens query scene tokens through cross-attention while no complementary channel lets scene elements assign constraint responsibility back to the trajectory hypotheses. This misaligns with the joint multi-agent formulation they adopt, in which every agent's future should be conditioned by all others symmetrically. To address this inductive-bias mismatch, this paper introduces interaction-symmetric decoding, a design principle that recasts scene-trajectory coupling as a dual assignment problem, pairing forward demand assignment (what scene information each trajectory hypothesis retrieves) with reverse constraint-responsibility assignment (which trajectory hypotheses each scene element should influence). We instantiate it in SymDiff Planner, whose Interaction-Symmetric DiT decoder couples a Demand-Responsibility Assignment module that realizes the assignment dual with an Adaptive Symmetry Modulation module that regulates the reverse assignment strength by scene context. Extensive experiments on the large-scale nuPlan benchmark show that SymDiff Planner achieves state-of-the-art closed-loop performance among learning-based planners, with the most pronounced gains on the challenging Test14-hard split dominated by dense agent interactions and long-tail driving cases. Code and pre-trained checkpoints will be released upon acceptance.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous Driving, Trajectory Planning, Diffusion Models, Imitation Learning, Multi-Agent Interaction, Attention Mechanisms

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NNeKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SymDiff Planner, which extends a diffusion-based trajectory planner with a reverse attention assignment branch and a conditional gate controlling its intensity. The design aims to complement standard trajectory-to-scene attention with scene-to-trajectory assignment. Evaluation on nuPlan includes closed-loop comparisons, component ablations, and plug-in experiments across three planner backbones.

Critical strengths
  • The closed-loop evaluation covers both non-reactive and reactive traffic across three nuPlan splits. The improvements, particularly on Test14-hard under non-reactive evaluation, suggest that the decoder modification has practical value.
  • The plug-in experiments report gains across regression, flow-matching, and diffusion planners. This suggests that the proposed module may be useful beyond a single backbone.
Critical weaknesses
  • The motivation for interaction symmetry is not sufficiently established. The paper argues that one-sided cross-attention conflicts with joint multi-agent generation, but joint modeling does not inherently require symmetric assignment. The proposed attention branch and gate are intuitively reasonable, yet the analysis does not clearly establish what useful property this symmetry provides beyond an alternative way of aggregating scene features. Its connection to diffusion modeling is also unclear.
  • The experiments don't isolate the claimed source of improvement. The ablations show benefits from adding the modules, but lack capacity-matched comparisons, such as an additional standard cross-attention branch. It remains unclear whether the gains come from the proposed assignment direction, additional parameters, or stronger scene conditioning.
Questions for authors
  • Could you clarify why joint multi-agent generation needs symmetric assignment, and what benefit the proposed design provides over standard cross-attention? Is the argument specific to diffusion, or is this a general attention modification?
  • Have you compared models with a similar number of parameters that differ only in which axis the attention weights are normalized over? These would help determine whether the gains support the symmetry motivation rather than additional model capacity.
  • How is the “ASM only” variant implemented in Table 3, given that ASM is defined as a gate on the reverse branch introduced by DRA?
1 comment

We thank the reviewer. All changes and additions are included in the revised PDF. The revised version renumbers some items; your Table 3 corresponds to Table 4 here, and all references below use the revised numbering.

Comment 1 (W1/Q1): Why symmetry, what benefit, and diffusion-specificity

  • Answer: The claim is not that joint modeling requires a symmetric update of both modalities; "symmetry" refers to the assignment direction. Standard cross-attention aggregates scene features into trajectory tokens and joint self-attention lets agents interact, but neither lets a scene element decide which trajectory hypotheses it should constrain. The reverse direction provides this scene-initiated assignment with an independent contribution: disabling the reverse branch lowers Test14-random from 91.30 to 89.76 (Figure 5), a 1.54 gap above the 0.36 run-to-run noise. The operator is a general attention-level modification, supported by the plug-in study across regression, flow-matching, and diffusion backbones (Table 3), while the controlled variants (Table 2) show it is not additional capacity. What is diffusion-specific is that the assignment is recomputed at every denoising step.
  • Evidence: Table 2; Table 3; Figure 5.
  • Boundary: We do not claim the assignment is necessary for every generative planner, only that it is the missing direction in the decoders we study.
  • Revision: Section 2.3 clarifies the assignment-versus-update distinction, cites Slot Attention, and states the generality and the diffusion-specific per-step coupling.

Comment 2 (W2/Q2): Capacity and normalization-axis controls

  • Answer: Yes. In Table 2 we compare variants of matched parameter count that differ only in the normalization axis. Scene-axis attention (6.96M) reaches 88.62 on Val14 against 90.56 for IS-DiT at the same 6.96M; a parameter-matched extra cross-attention branch (6.93M) reaches 89.04; Slot attention (6.82M) 88.22, and all three splits show the same ordering. The gain therefore tracks the scene-initiated, trajectory-axis assignment rather than additional capacity or stronger scene conditioning.
  • Evidence: Table 2, including its #Params column.
  • Boundary: All variants are trained from scratch with the same protocol.
  • Revision: Table 2 (Controlled decoder variants) added in Section 5.2.

Comment 3 (Q3): Implementation of the ASM-only variant

  • Answer: In Table 4, ID 2 is not ASM. Since ASM gates the reverse branch introduced by DRA, it cannot exist without DRA; the only implementable variant gates the forward cross-attention residual instead (marked with a dagger) and is not used to measure ASM, whose contribution is measured by ID 1 to 3 (+0.99). Table 6 then gives the exact step-3 computation of all four variants.
  • Evidence: Table 4 (IDs 0-3, dagger note); Table 6.
  • Revision: Table 4 dagger note; Table 6 (Appendix A.4).

We will promptly address any further questions the reviewer may have.

Reviewer_g3CDOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SymDiff Planner, a diffusion-based framework for closed-loop autonomous driving that modifies how the decoder interacts with scene representations. The proposed Demand-Responsibility Assignment module combines conventional trajectory-to-scene cross-attention with an additional assignment mechanism that distributes scene information across trajectory tokens. Adaptive Symmetry Modulation introduces a conditional gate to regulate this additional branch. The authors interpret this construction as an interaction-symmetric inductive bias for joint ego planning and neighboring-agent prediction. The method is evaluated on three nuPlan splits in non-reactive and reactive settings, with additional experiments involving rule-based refinement, integration into other planning backbones, component ablations, and qualitative examples. The reported non-reactive improvements are promising, particularly on Test14-hard. However, the central motivation, component attribution, and reliability of several reported results require substantial clarification.

Critical strengths
  1. The paper addresses a relevant problem and evaluates actual closed-loop behavior. Improving planning under interactions with surrounding agents is practically important. Reporting both non-reactive and reactive closed-loop results is more informative than relying only on open-loop trajectory accuracy.

  2. The proposed architectural modification is compact, and its main operations are reasonably clearly described. A useful interaction module that can be incorporated into existing planners would have practical value.

  3. Some reported improvements are encouraging. In Table 1, the standalone model achieves 78.04 NR-CLS on Test14-hard, compared with 75.99 for Diffusion Planner and 76.47 for Flow Planner. The Test14-random NR-CLS result is also favorable. These results motivate further investigation, subject to resolving the reporting and statistical concerns below.

Critical weaknesses
  1. The characterization of the closest prior work is inaccurate and undermines the central motivation. Section 2.3, particularly lines 120-127, states that both Diffusion Planner and Flow Planner use one-sided cross-attention with fixed scene tokens. However, the original Flow Planner paper, Section 3.2, Equations (4)-(6), concatenates lane, neighboring-agent, and ego-trajectory representations, applies joint self-attention, and subsequently updates the modality-specific features. This is materially different from the fixed-scene, trajectory-query-only architecture described in this submission. Its trajectory tokenization is also presented in terms of ego-trajectory segments, rather than the same joint ego-and-neighbor future-generation formulation assumed here. This discrepancy is consequential because the paper's main contribution is framed as identifying and repairing an interaction pathway missing from existing generative planners. The authors need to distinguish their particular assignment mechanism from existing joint fusion mechanisms and accurately explain the remaining limitation. The discrepancy also makes the Flow Planner plug-in experiment difficult to interpret, since the insertion point and preserved components of the original architecture are not specified.

  2. The numerical evidence in Table 2 is inconsistent under the apparent averaging convention. The caption describes the reported average as the mean over the selected scenarios. However, taking the arithmetic mean of the five displayed category scores does not reproduce two of the reported baseline averages. For PLUTO, the five baseline scores average to 88.654, rather than the reported 87.43. The corresponding augmented-model average is 90.85. Therefore, the improvement implied by the displayed scores is approximately 2.20 points, rather than the reported 3.42 points. For Flow Planner, the five baseline scores average to 88.814, rather than the reported 89.15. The corresponding augmented-model average is 91.10. Therefore, the improvement implied by the displayed scores is approximately 2.29 points, rather than the reported 1.95 points. Both reported gains are repeated in the main text at lines 376-377. The Diffusion Planner rows, in contrast, agree with the arithmetic calculation. These discrepancies exceed what rounding could explain. If the authors instead use scenario-count weighting, the weights and aggregation procedure must be provided and applied consistently to each paired comparison. As written, the table does not permit verification of a central claim about generality across backbones.

  3. The ASM-only ablation is not defined by the proposed computation. According to Section 4.2, ASM controls only the reverse branch introduced by DRA. If DRA is removed, the reverse branch is absent, and the described ASM gate has no remaining contribution to modulate. Nevertheless, Table 3 reports a distinct model without DRA but with ASM, with a score of 88.16 compared with 87.52 for the baseline. Figure 7 likewise attributes behavioral improvements to this variant. An alternative implementation of ASM without DRA may be possible, but it is not specified. For example, applying ASM to the forward branch would constitute a different intervention from the ASM defined in the method. Consequently, the current four-way ablation does not establish the claimed individual contributions or complementarity. Exact computation graphs or pseudocode for all four variants are necessary.

  4. The claimed symmetry principle is insufficiently justified, and the architectural novelty is not adequately isolated. The joint representation of multiple agents' futures does not, by itself, establish that the conditional distribution requires two directions of scene-trajectory assignment. Joint modeling, permutation symmetry, and normalization over different attention dimensions are distinct concepts. The ego vehicle and neighboring agents also have different roles in the planning task. Furthermore, conventional cross-attention already transfers scene information into trajectory representations, while trajectory self-attention allows generated agents to influence each other. The paper does not identify a precise representational limitation that these existing operations cannot address. At the operator level, DRA introduces an additional attention map normalized across output tokens and uses it to aggregate input features. Related normalization already appears in Object-Centric Learning with Slot Attention by Locatello et al., published at NeurIPS 2020, where input features compete for assignment to output slots, followed by weighted aggregation. The proposed module differs in its aggregation and integration into the planner, but this relationship should be discussed when establishing novelty. The current experiments do not distinguish the proposed principle from the effects of extra parameters, an additional feature aggregation path, or changed normalization. An additional conventional cross-attention branch with a comparable parameter count, together with a reverse branch using a fixed weight, would provide informative controls. The method may be a useful architectural modification, but the stronger characterization as a previously missing symmetry principle is not yet supported.

  5. The interpretation of ASM and the diagnostic experiments exceeds what the implemented mechanism demonstrates. Section 4.1 states that the input to ASM combines the navigation-route encoding and the diffusion-time embedding. Therefore, when the route representation and diffusion timestep are held fixed, changing surrounding vehicles or their interactions does not change the ASM gate. The reverse attention itself can respond to these changes through its inputs, but the ASM gate cannot directly do so. Appendix A.5 acknowledges this proxy limitation. Nevertheless, the main text repeatedly interprets ASM as adapting coupling strength to a scene's interaction requirements without directly demonstrating that behavior. The interpretation of Figure 5 is also problematic. A larger trajectory displacement after masking an entire object category measures sensitivity to that intervention. It does not establish more accurate interaction reasoning, appropriate constraint assignment, or improved safety. Increased displacement could also reflect brittleness under a large input perturbation. Because the planner is generative, the comparison additionally requires identical initial noise for the original and masked conditions. This control is not described. Finally, Table 4 attenuates both the forward and reverse contributions through their shared residual gate. The resulting degradation establishes the importance of the overall scene-conditioning residual, but does not isolate the value of reverse assignment or adaptive modulation.

Questions for authors
  1. How do the authors reconcile their description of Flow Planner with its joint fusion mechanism in Section 3.2 of the original paper? Where exactly is DRA inserted in the original Flow Planner architecture, and which original components are retained?

  2. Can the authors provide the aggregation code or underlying results for Table 2 and explain the discrepancies identified above? Please also reconcile Table 3's full-model score of 89.93 with Table 1. Using the stated pool size of 1,654 and split sizes of 1,118 and 272, with an implied remaining 264 scenarios, straightforward scenario-weighted pooling of Table 1 gives approximately 88.62. Were different runs, checkpoints, scenario memberships, or aggregation conventions used?

  3. What is the exact computation performed by the ASM-only variant? Are all variants trained from scratch or initialized from a pretrained planner? If the latter, how does initializing the shared cross-attention residual gate to zero preserve the pretrained computation when this also suppresses the original forward cross-attention contribution?

  4. What results distinguish DRA from an additional conventional cross-attention branch with a comparable parameter count? How much improvement remains when the reverse-branch weight is fixed or learned as a global scalar, rather than conditioned on route and diffusion time?

5 comments

In fact, I am willing to discuss this with the authors and may improve my score based on the rebuttal content if it really convinces me. But I have not received any feedback from the authors.

Dear Reviewer,

Thank you for the review. The additional analyses are underway and the manuscript has not yet been revised. For each point, we first state our thinking together with the evidence already in the manuscript, then the analysis being added and the criterion by which it will be judged.

W1, Q1. Flow Planner and the plug-in

  • Thinking: the difference we target is the assignment interface, not information flow. DRA provides a distribution over hypotheses for each scene element, which joint self-attention does not express.
  • Evidence: both cross-attention and joint self-attention update each token by normalizing over the set it attends to, so neither yields a per-scene-element distribution over hypotheses.
  • Underway: a new appendix table with three configurations, Flow Planner, Flow Planner + DRA with ego-only tokens, and joint-flow + DRA, with DRA inserted after the cross-fusion step and all original components retained. A gain under both representations would attribute the effect to DRA rather than to representation or joint fusion.
  • Revision: Section 2.3 clarification; Section 5.2 insertion point.

W2, Q2. Table 2 average

  • Thinking: Avg. is computed over all 14 Val14 categories; the five displayed columns are a representative subset.
  • Evidence: the displayed columns therefore do not sum to Avg., and the aggregation is identical for both rows of each pair.
  • Underway: the caption will state this, and a new appendix table will give the 14-category breakdown together with the aggregation script.
  • Revision: caption and appendix table.

Q2. Table 3 versus Table 1

  • Thinking: Table 3 is one pooled evaluation on a merged list built through a single construction path, so it need not equal the weighted average of the official splits.
  • Evidence: both use the same checkpoint and inference configuration; the merged list differs from the split union in construction, since Test14-random samples 20 scenarios per category.
  • Underway: re-evaluation of the four variants on the fixed 1,654-scenario list, reported per split and pooled. If the pooled value reproduces, the gap is explained by the path; if not, Table 3 will be corrected.
  • Revision: Table 3 and appendix table.

W3, Q3. ASM-only variant and initialization

  • Thinking: ID 2 is not ASM. It gates the forward residual, a different intervention; ASM proper is measured by ID 1 to ID 3.
  • Evidence: the new appendix computation graph specifies the gated branch, projection sharing, normalization axis and initialization for IDs 0-3. All variants are trained from scratch with zero-initialized new projections, so the dual-assignment step writes nothing at the first step.
  • Revision: Table 3 footnote, Section 5.2, and the caption and interpretation of Figure 7.

W4, Q4. Symmetry, novelty, controls

  • Thinking: the property is competition among hypotheses for each scene element, which the forward path does not create.
  • Evidence: forward attention is query-normalized, while the reverse path normalizes over hypotheses per scene element (Eq. 9).
  • Underway: a seven-row controlled table, Standard DiT, extra cross-attention with matched parameters, scene-axis normalization, slot attention, fixed gate, learned-scalar gate and IS-DiT; an appendix sweep over slot-attention iterations; and an analysis of per-scene-element entropy and concentration. The claim is supported if the gain disappears under the capacity-matched and axis-swapped controls, and if the reverse maps are sparser than the forward maps.
  • Revision: Section 5.2 and Section 2.3.

W5. ASM interpretation, Figure 5, Table 4

  • Thinking: the gate is route- and timestep-conditioned by construction; interaction dependence enters through the reverse attention.
  • Evidence: Figure 5 measures sensitivity, not accuracy, and Table 4 scales both branches jointly.
  • Underway: the masking analysis is re-run with shared initial noise and a routine-category control; a branch-wise sweep over forward and reverse residuals replaces Table 4. A steeper degradation under reverse-only scaling would isolate the reverse branch.
  • Revision: Figure 5, Table 4 and the ASM description.

We thank the reviewer for the detailed audit. All changes and additions are included in the revised PDF. The revised version renumbers several items: your Table 2, Table 3, Table 4, Figure 5, and Figure 7 correspond to Table 3, Table 4, Figure 5, Figure 6, and Figure 8 here.

Comment 1 (W1/Q1): Flow Planner description and DRA insertion

  • Answer: We corrected the characterization. Diffusion Planner couples scene and trajectory through one-sided cross-attention; Flow Planner tokenizes the ego trajectory and fuses lanes, neighbors, and ego tokens with joint self-attention (Eqs. 4-6 of its paper), generating the ego trajectory only. Neither exposes an explicit scene-to-hypothesis assignment, which is what DRA adds. In Table 8, DRA is inserted after Flow's cross-fusion step, leaving its tokenizer, joint self-attention, decoder, and flow-matching head unchanged, and raises Flow ego-only from 90.43 to 91.24 and joint from 90.48 to 91.28 on Val14.
  • Evidence: Table 8.
  • Boundary: Flow Planner is ego-only, so the joint setting is our controlled construction.
  • Revision: Section 2.3 rewritten; Table 8 (Appendix A.6) specifies the insertion point.

Comment 2 (W2/Q2): Aggregation and table consistency

  • Answer: Avg in Table 3 is the scenario-count-weighted score over the full Val14 split (Eq. 14), not the mean of the five displayed categories, which are a representative subset. Table 7 lists all 14 categories with counts (sum 1,118), so each Avg reproduces as ∑cncsc/∑cnc\sum_c n_c s_c / \sum_c n_c; corrected gains are PLUTO +3.40, Flow +1.96, Diffusion +3.68. Table 4's 89.93 is a single evaluation over the union pool (1,654 scenarios) while Table 1 reports three per-split scores; since Test14-random is a randomly sampled 264-scenario subset, the pooled and per-split evaluations draw different scenario memberships, so count-weighted pooling of Table 1 is not expected to reproduce the pooled score.
  • Evidence: Table 7 (14 categories and counts), Eq. 14, Table 3.
  • Boundary: The discrepancy stems from the scenario membership of the randomly sampled Test14-random subset, not from a different model or checkpoint.
  • Revision: Table 3 caption and Section 5.2 numbers; Table 7; Eq. 14; Table 4 caption.

Comment 3 (W3/Q3): ASM-only variant and computation graphs

  • Answer: In Table 4, ID 2 is not ASM. Since ASM gates the reverse branch introduced by DRA, it cannot be instantiated without DRA; the only implementable variant gates the forward cross-attention residual instead (marked with a dagger), which is a distinct intervention not used to measure ASM, whose contribution is measured by ID 1 to 3 (+0.99). Table 6 gives the exact step-3 computation of IDs 0-3, and all variants are trained from scratch, so zero-initializing γcross\gamma_{\mathrm{cross}} suppresses no pretrained computation.
  • Evidence: Table 6 gives the exact step-3 computation of IDs 0-3; Table 4: ID 0 to 1 +1.42, ID 1 to 3 +0.99.
  • Revision: Table 4 dagger note; Table 6 (Appendix A.4); Section 4.2 and A.1.3 state from-scratch training.

Comment 4 (W4/Q4): Novelty versus capacity, Slot Attention, and gating

  • Answer: We do not assume agent-level symmetry; the ego and neighboring agents keep their distinct roles, and "symmetry" concerns only the scene-to-hypothesis assignment direction, not a permutation over agents. In Table 2 the gain is not from capacity, an alternative aggregation, or Slot Attention: a parameter-matched extra cross-attention branch reaches 89.04 against 90.56 for IS-DiT on Val14; scene-axis attention 88.62; Slot attention 88.22; replacing the conditioned gate by a constant (α=1\alpha = 1) gives 90.34 and by a global scalar 90.24, i.e., 0.22 and 0.32 below IS-DiT. Slot Attention normalizes over output tokens as DRA does, but DRA uses private WQ,WKW^Q, W^K, a single pass, and a gate α(y)\alpha(y) decoupled from the residual gate.
  • Evidence: Table 2.
  • Boundary: Both are attention-level operators; the architectural differences above are confirmed by the controlled comparison in Table 2.
  • Revision: Section 2.3 discusses Slot Attention and generality; Table 2.

Comment 5 (W5): ASM interpretation and diagnostic figures

  • Answer: We clarify that ASM is conditioned on the route and diffusion step (sample-conditional), not on scene interaction. The masking study (Figure 6) is reported as input sensitivity with identical initial noise, a routine null control, and a critical versus non-critical split, not as evidence of better reasoning. The shared-gate issue is addressed by scaling the branches independently (Figure 5): removing the reverse branch lowers Test14-random from 91.30 to 89.76, a 1.54 gap above the 0.36 run-to-run noise.
  • Evidence: Figure 6 (critical vs non-critical p=2.4×10−38p = 2.4\times10^{-38} and 3.7×10−33.7\times10^{-3}); Figure 5.
  • Boundary: ASM's gate depends on route and timestep, not on individual agents, which we state explicitly.
  • Revision: ASM wording in abstract, intro, Section 4.2, and conclusion; Figures 5 and 6.
Reviewer_Hy8vOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses trajectory planning by modifying the cross-attention mechanism in Diffusion Planner. It introduces two modules: the demand responsibility assignment and the adaptive symmetry modulation. The demand responsibiltiy assigment augments standard cross-attention with a reverse assignment branch, in which each scene token assigns weights to trajectory tokens. These weights are then used to aggregate scene features into the trajectory representations. The adaptive symmetry modulation produces a scalar conditioned on route information and the diffusion timestep to control the contribution of this reverse branch.

Critical strengths
  • The paper is clearly written, and the overall framework is easy to follow.
  • The proposed modification is relatively simple and can be incorporated into existing planners.
Critical weaknesses

1. Unclear novelty and motivation

The paper argues that the cross-attention mechanism used in existing methods is misaligned with joint multi-agent trajectory generation. However, standard cross-attention already transfers scene information to trajectory tokens, while self-attention enables interactions among trajectory tokens. It is therefore unclear why the existing architecture is fundamentally insufficient for modeling joint trajectories, or how the proposed modification addresses a specific limitation.

Furthermore, normalizing attention over the receiving tokens is related to existing mechanisms such as Slot Attention [1]. The manuscript should clarify this relationship and provide controlled comparisons to establish whether the improvements arise from the proposed assignment mechanism or simply from additional attention capacity.

2. Reliability of the reported performance improvements

Several improvements in Table 1 are small, and the proposed method does not consistently outperform the strongest baseline. Given that improved interaction modeling is a central motivation of the paper, these results warrant further analysis.

The authors should report means and standard deviations across multiple independent runs, distinguishing variability due to training from variability due to inference-time sampling. This would help establish whether the reported improvements are consistent and support the paper’s central claims.

[1] Locatello et al., “Object-Centric Learning with Slot Attention,” NeurIPS 2020.

Questions for authors
  1. How does the reverse assignment mechanism differ from Slot Attention, and what benefit does this difference provide for trajectory planning?
  2. Could the proposed method outperform a baseline model which simply adds standard cross-attention layers with a similar number of parameters?
  3. Are the improvements in Table 1 consistent across different training and sampling seeds? Please report the mean and standard deviation.
2 comments

We thank the reviewer. All changes and additions are included in the revised PDF. We address (1) the novelty of the reverse assignment and its relation to Slot Attention, (2) the capacity-controlled comparison, and (3) the reliability of the reported gains.

Comment 1: Novelty and the relation to Slot Attention

  • Answer: Standard cross-attention transfers scene information to trajectory tokens and self-attention lets trajectory tokens interact, but neither lets a scene element assign constraint responsibility to specific trajectory hypotheses, which is what the reverse assignment adds. The one-sided decoder underperforms, with the gap largest on the interaction-heavy split: 89.87 vs 90.56 on Val14 and 75.99 vs 78.04 on Test14-hard (Standard DiT, Table 2). The gain is not from capacity or an alternative mechanism: in Table 2 an extra cross-attention branch (6.93M) reaches 89.04, scene-axis attention 88.62, and Slot attention 88.22, all below IS-DiT, and sweeping Slot-attention iterations K=1K=1 to 5 leaves the gap unchanged (88.46 at K=5K=5). The operator is structurally close to Slot Attention in normalizing over the receiving tokens, but differs by private WQ,WKW^Q, W^K, a single assignment pass integrated with the forward branch, and a gate α(y)\alpha(y) decoupled from the residual gate. Removing the reverse branch lowers Test14-random from 91.30 to 89.76 (Figure 5).
  • Evidence: Table 2; Figure 5; Section 5.2 (KK-sweep).
  • Boundary: Both are attention-level operators; the architectural differences above are confirmed by the controlled comparison in Table 2.
  • Revision: Section 2.3 reframes one-sided cross-attention and joint self-attention as both assignment-free and cites Slot Attention; Table 2 (Section 5.2) reports the controlled variants.

Comment 2: Would added standard cross-attention layers of similar size match the method?

  • Answer: No. In Table 2, a parameter-matched extra standard cross-attention branch (6.93M) increases capacity without adding an assignment direction and stays below IS-DiT (6.96M) on all three splits: 89.04/76.12/88.33 vs 90.56/78.04/91.32.
  • Evidence: Table 2.
  • Boundary: We add one matched branch; stacking further layers raises parameter count without introducing a scene-initiated channel, so we do not expect it to close the gap.

Comment 3: Are the improvements consistent across training and sampling seeds?

  • Answer: Without refinement the method leads the strongest learning-based baseline on all three splits (90.56 vs 90.43, 78.04 vs 76.47, 91.32 vs 89.88; Table 1), so it does consistently outperform it. Relative to Diffusion Planner, its backbone, the gains exceed the training-seed std (Table 9) on Test14-hard (2.05 vs 1.48) and Test14-random (2.13 vs 1.87) while Val14 (0.69 vs 0.78) is within it, and training variance dominates inference sampling (std 0.15/0.30/0.37).
  • Evidence: Table 9: over three training seeds Val14 90.87 ±\pm 0.78, Test14-hard 77.00 ±\pm 1.48, Test14-random 89.64 ±\pm 1.87.
  • Revision: Table 9 (Appendix A.7) reports the variance decomposition; Section 5.1 reports the seed-level comparison.

We will promptly address any further questions the reviewer may have.

Reviewer_Hy8vOpenReview ↗

Thank you for the reponse

Thank you for the detailed response and the additional experiments.

The clarification on how the method differs from Slot Attention and the controlled variants in Table 2 are helpful. The parameter-matched baseline addresses part of my concern. That said, each variant in Table 2 was run only once, and most of the differences on Test14-hard and Test14-random are within the training-seed standard deviations reported in Table 9. I still have a few concerns, e.g.,

  • Seed variance. Based on the three-seed means in Table 9, the gaps relative to Flow Planner, the strongest baseline, are within one standard deviation on all three splits (+0.44, +0.53, and −0.24).
  • Baseline comparison. The gains of 2.05 and 2.13 cited in Section 5.1 are relative to Diffusion Planner, not Flow Planner.
  • Reactive setting. I’m still not sure why the method doesn’t outperform the baselines on Test14-hard R-CLS

I’d encourage the authors to report results across multiple seeds for their method, the main baselines, and the Table 2 variants. That would make the reported gains more convincing.

Given these points, I’ll keep my score unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.