- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper introduces a non-commutativity diagnostic to the analysis of DiT attention heads, proposing an additive aggregation of heads while noting that the state-dependent residual operators do not commute; experiments are conducted using DiT-XL/2 on ImageNet 256×256, with transfer to CUB-200-2011.
Critical strengths
- Introducing non-commutative/operator-order analysis into DiT attention heads is a novel approach.
- The design is ingenious: Phase 1 diagnoses and selects head pairs, while Phase 2 applies regularization only to the selected pairs.
- The FID improvements at Layer 24 and the joint 10+24 configuration are significant and convincing.
Critical weaknesses
- The core diagnosis relies on the sequential application of single-head updates, whereas standard DiT models aggregate outputs via addition after computing all heads in parallel. Does the identified sensitivity to operation order actually reflect non-commutative effects present in truly parallel models?
- There is scant discussion regarding the selection of key hyperparameters, potential overfitting, or result sensitivity, eg. concerning the layer-specific scale in Phase 1, the pair scale in Phase 2, and the techniques detailed in Appendix A.6 (such as online calibration, EMA, log-space centering, and clipping).
- For Phase 1 graph energy, Layers 3, 10, and 24 were selected, but the NC effect at Layer 3 was nearly identical to the "no-NC" baseline, demonstrating that high graph energy does not guarantee effective intervention. Nevertheless, Layer 3 was still chosen as an intervention layer, resulting in wasted computation and undermining the credibility of the selection criteria.
- Table 2 reveals a significant disparity in FID scores between the "no-NC" and NC settings, it is unclear whether the "no-NC" setting also underwent the same partial fine-tuning process.
- The scope of the experiments and the assessment of generalizability are too limited.
Questions for authors
see weakness
Dear Area Chair and Reviewers,
We would like to draw your attention to a point of continuity in the discussion. Reviewer 2X68’s Comment 2 raises a concern that we addressed in our earlier response to Reviewer sEEv, which covers the magnitude-ranking comparison, expanded-pool random selection, and the distinction between pair selection and the hinge–shrinkage comparison.
To ensure this prior response is fully considered, we have reiterated the relevant findings in our direct reply to Reviewer 2X68. We respectfully ask the Area Chair to review the linked exchange alongside our latest reply and to draw the other reviewers’ attention to this evidence. If any aspect of the concern remains unresolved after these responses and experiments, we would welcome clarification of the specific point still at issue, so that the discussion can focus on any remaining disagreement.
Kind regards,
The Authors