acceptodds
Withdrawn from ICLR 2027

Counterfactual Discovery of Motion-Effect Circuits in Video Diffusion Transformers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video diffusion transformers make camera and object motion observable, but observation and attribution do not by themselves identify which internal computations support a particular motion effect. Large attribution can reflect denoising shared across motions rather than support specific to the requested direction. We introduce CAP-MED, a counterfactual activation-patching protocol for motion-effect node discovery. It uses target-vs-other attribution to select compact node sets across denoising steps, layers, and submodules, then tests whether exchanging their activations recovers and suppresses the effect between matched motion-instructed and empty-action rollouts. On Wan2.2, specificity-selected sets achieve median normalized recovery of 0.99 and suppression of 0.97, compared with 0.15 and 0.59 for attribution-magnitude selection at the same node budget. Selector controls and independent camera and object readouts support the motion-specific interpretation of these effects. Further head-level interventions identify conditional dependence between selected sets and a value-dominant effect within selected camera-pan heads. LTX-Video experiments test the full workflow on a second architecture and extend it to color temperature. The evidence characterizes selected node sets and their functional roles rather than a complete motion-generation circuit.

Primary area:
interpretability and explainable AI
Keywords:
video diffusion, interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_H8zpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CAP-MED is a protocol for finding sets of internal activations in a video diffusion transformer that support a specific motion effect. For a scene prompt and seed, an instructed rollout V1V_1 (e.g. "pan right") and an empty-action rollout V0V_0 are generated; the motion effect is the change in a VideoMAE classifier logit difference. A node is the full token-by-channel output of a submodule (SA/FFN/CA) at one layer and one denoising step. EAP-IG edge attribution is summed per layer, and layers are ranked by target-vs-other specificity (target attribution divided by the mean attribution to the other nine signed directions); the top-6 layers x steps {0,2,4} x {SA, FFN} give a 36-node set. Sufficiency inserts V1V_1's activations into the V0V_0 rollout at those nodes and measures recovery of the effect; necessity does the reverse. On Wan2.2 T2V-A14B (10 steps, 40 layers) over ten signed camera/object directions, specificity-selected sets reach median normalized recovery/suppression 0.99/0.97 versus 0.15/0.59 for the top-36 nodes by raw attribution magnitude; same-grid random-layer and high-attribution/low-specificity controls and a linear-probe selector do worse. Independent readouts (VBench CameraPredict, CoTracker3, dense flow) preserve the ordering. Head-level experiments show a two-set composed dependence and that exchanging V (not QK) in pan heads carries the effect. On LTX-Video the workflow is rerun with a per-node magnitude-balanced variant for camera motion and color temperature, with mixed per-direction results.

Critical strengths

1. An extensive and well-controlled intervention protocol. Matched-pair bidirectional patching, same-grid controls that separate layer choice from allocation, a probe-based selector, readouts not used for selection, reference-based and reference-free quality metrics, paired bootstrap intervals, budget/step/layer sweeps, and a direct CA-addition test. Mechanistic analysis of video DiTs is underexplored and the protocol is reusable; the appendix is unusually complete.

2. Scope is stated carefully. The paper repeatedly distinguishes what the exchanges show (functional dependence of a measured score) from what they do not (a minimal circuit, content-preserving editing).

Critical weaknesses

1. The intervention unit is so coarse that near-complete recovery is expected, and the main baseline is weak. A "node" is the entire output field of SA or FFN at a layer and step; the selected set transplants 6 layers x 2 submodules at 3 of the 10 steps, i.e. a large slice of the early residual-stream computation, including appearance (LPIPS 0.51 from patched to the empty-action video; acknowledged in App. E). At that granularity ρ≈1\rho \approx 1 is not surprising: Table 20 shows that any three late layers (37 to 39) at those steps already give ρ=1.00\rho=1.00, that step-0 nodes are useless (ρ=0.03\rho=0.03) and step 2 alone gives 0.99. The headline contrast is with "EAP-IG top-36 individual nodes", whose failure is largely an allocation artefact (it spreads nodes over steps and layers where patching does nothing). The same-grid controls show that layer choice on the grid matters, but not that the specificity ratio is the right criterion: the obvious heuristics are never compared (last-k layers; top-k by target magnitude on the same grid; a step sweep). What is learned about the model, beyond "camera motion is set in mid/late layers at steps 2 to 4 and carried by V in pan heads", is thin for an interpretability paper, and the authors disclaim circuit-level conclusions.

2. The central claim (specificity beats magnitude) does not replicate cleanly outside Wan2.2 camera motion. On LTX-Video the selector had to be changed to a per-node magnitude-balanced score, raw EAP-IG has larger median effects in both pan directions and both arms (Table 25), sign-correctness orderings vary by cell, and the pooled numbers carry no uncertainty. On Wan2.2, object-lateral specificity ratios are below one and effects are weak until the budget is doubled, and object-depth sufficiency is partial. The abstract's numbers therefore describe a Wan2.2 camera-motion result obtained with a selector whose grid (steps {0,2,4}, SA/FFN, K=6) was chosen on that model; generality is asserted more than shown.

Presentation note. Many tables, several distinct selected sets (36-node grid, K=100 head sets, top-300 head set, LTX per-node sets) and dense caveats obscure the two or three takeaways. Section 4.7 opens with a sentence lacking a subject ("Instead of the Wan layer-grid rule, it ranks...").

Questions for authors

Q1. At the same 36-node budget and the same step/submodule grid, how do these compare with specificity ranking: the last six layers; the six layers with highest target-attribution magnitude; six layers chosen by a leave-one-out patching sweep? If a trivial heuristic matches specificity, the contribution reduces to the protocol.

Q2. How much of the recovery is appearance transfer? Please report recovery as a function of content change (e.g. patch a random fraction of tokens or channels within the selected fields, or report ρ\rho per unit LPIPS), or restrict the exchange to a motion-relevant subspace.

Q3. Please add paired uncertainty for the LTX full-workflow comparison (Tables 7 and 25) and repeat attribution with different discovery pairs. Is the specificity advantage statistically supported anywhere on LTX?

Q4. In Eq. (4) the "other" set includes the opposite direction on the same axis, whose attribution to Ya=la−laˉY_a = l_a - l_{\bar a} is sign-correlated with the target's. How do the selected layers and results change if the opposite direction is excluded from the denominator?

Q5. The classifier readouts were trained on Wan2.2 generations annotated by reviewers who saw the instruction. Dense-flow directional agreement on unpatched instructed videos is only 62 to 68% for object motion. For object directions, how should the reader interpret ρ\rho when the ground-truth motion itself is ambiguous in a third of cases?

5 comments
AuthorsOpenReview ↗

Official Comment for Reviewer H8zp (1/4)

We thank the reviewer for the thorough and constructive review, and for recognizing the extensive and well-controlled intervention protocol, its reusability, the completeness of the appendix and our careful statement of scope. The suggested controls were especially valuable. Following them, we report the requested same-grid heuristics, content measurements across all ten directions, paired uncertainty and repeated discovery on LTX-Video, and the opposite-exclusion check. We address each point below and hope that these responses resolve the reviewer's concerns.

W1 + Q1. Identical-grid controls and scientific contribution

We thank the reviewer for this detailed critique and for proposing these controls. We ran all three on the identical grid.

Selector; medians, N=40N=40Sufficiency ρ\rhoNecessity ρ\rho
SPEC1.00111.00110.98930.9893
Target magnitude−0.0169-0.01690.02430.0243
Last-six1.00251.00251.00891.0089
LOO; one discovery pair0.43000.43000.51700.5170

All selectors share the original pairs and the six-layer {0,2,4}\{0,2,4\} SA/FFN grid. Sufficiency inserts instructed activations into the empty-action run, and necessity exchanges in the opposite direction. The leave-one-out sweep used 1,680 forward passes, 1,640 for selection and 40 for endpoints.

Coarse units do not make recovery automatic. At the same 36-field budget and grid, the six layers with the largest target attribution give median ρ\rho of −0.02-0.02 and 0.020.02. Fixed three-layer sets at the same steps give −0.02-0.02 for layers 15 to 17 and −0.06-0.06 for layers 5, 22 and 35 (Table 20). Near-complete recovery therefore depends on which layers are exchanged, not on the size of the exchanged slice. The failure of attribution ranking is also not an allocation artefact, because it persists when allocation is fixed.

Specificity and the last six layers. Late layers are effective for Pan+, as our depth sweep showed (Table 20), and the last six layers match specificity there. Specificity reaches this effect from attribution alone, without a depth prior, and with a largely different set, since only layers 37 and 39 are shared. Across directions, specificity also finds effective sets away from the last layers. Its tilt+, tilt− and zoom− sets each share only one layer with layers 34 to 39 and reach median sufficiency ρ\rho of 0.880.88, 0.980.98 and 1.001.00 (Table 1). Specificity therefore does not reduce to a late-layer rule.

Patching sweep. The leave-one-out patching sweep, at 1,680 forward passes, selects a set with lower effect than specificity (0.430.43 and 0.520.52).

What is learned. Beyond where the effect is located, controlled exchanges show that it is carried mainly by attention values under the recipient's routing, that it persists only when later sites receive the matching state, and that patched motion follows its own donor's time course (median correlation 0.9550.955 versus 0.0200.020 for other sources).

In the revised paper, we will add these same-grid controls to the main results.

AuthorsOpenReview ↗

Official Comment for Reviewer H8zp (2/4)

Q2. Motion effect per perceptual change and reference fidelity

We thank the reviewer for this question. We answer it in four ways, including the effect per unit LPIPS that the reviewer suggested.

What can transfer. Each pair shares the scene prompt, seed and sampler and differs only in the appended instruction. A full-field exchange therefore cannot import unrelated appearance. It can transfer only differences caused by the instruction, which include the motion itself and its accompanying changes, such as newly revealed regions under camera motion.

The patched change tracks the instructed change.

Pooled sufficiency (App. E)CAP-MED patchedInstructed videoEAP-IG patched
Subject consistency change−0.038-0.038−0.043-0.043−0.017-0.017
Background consistency change−0.025-0.025−0.027-0.027−0.012-0.012
Aesthetic quality change−0.037-0.037−0.048-0.048−0.018-0.018
FVD to instructed reference328328443443

Changes are relative to the empty-action video. CAP-MED changes the video by about as much as the instruction itself does, and its outputs are closer to the instructed videos than EAP-IG outputs are. EAP-IG changes the video less because it also induces much less motion (median ρ\rho 0.150.15). The patched-to-empty LPIPS of 0.510.51 is therefore consistent with the instruction's own effect on the frames, since camera motion itself changes frame content.

Effect per unit LPIPS. On Pan+, the paired median gain of SPEC in ρ/LPIPS\rho/\mathrm{LPIPS} is +1.70+1.70 in sufficiency and +1.51+1.51 in necessity over same-grid magnitude, and +0.53+0.53 and +0.08+0.08 over global top-36.

Comparison with the last six layers. Sufficiency, forty original pairs per direction, mean (median).

DirectionDINO SPECDINO last-sixLPIPS SPECLPIPS last-six
Pan+0.5828(0.5940)0.5828(0.5940)0.5781(0.5744)0.5781(0.5744)0.5842(0.5783)0.5842(0.5783)0.5921(0.5844)0.5921(0.5844)
Pan-0.6049(0.6110)0.6049(0.6110)0.6137(0.6284)0.6137(0.6284)0.5768(0.5844)0.5768(0.5844)0.5680(0.5772)0.5680(0.5772)
Tilt+0.6713(0.6995)0.6713(0.6995)0.5485(0.5340)0.5485(0.5340)0.5185(0.5324)0.5185(0.5324)0.6222(0.6363)0.6222(0.6363)
Tilt-0.6432(0.6347)0.6432(0.6347)0.5401(0.5283)0.5401(0.5283)0.5403(0.5521)0.5403(0.5521)0.6378(0.6510)0.6378(0.6510)
Zoom+0.6798(0.6945)0.6798(0.6945)0.6271(0.6633)0.6271(0.6633)0.5037(0.5219)0.5037(0.5219)0.5492(0.5705)0.5492(0.5705)
Zoom-0.7175(0.7371)0.7175(0.7371)0.6528(0.6742)0.6528(0.6742)0.4788(0.4933)0.4788(0.4933)0.5437(0.5378)0.5437(0.5378)
Lateral+0.8081(0.8017)0.8081(0.8017)0.6992(0.7166)0.6992(0.7166)0.3892(0.3963)0.3892(0.3963)0.4855(0.4747)0.4855(0.4747)
Lateral-0.6907(0.7118)0.6907(0.7118)0.6723(0.6833)0.6723(0.6833)0.4715(0.4791)0.4715(0.4791)0.4988(0.5152)0.4988(0.5152)
Depth+0.7926(0.7987)0.7926(0.7987)0.6794(0.6998)0.6794(0.6998)0.3876(0.4027)0.3876(0.4027)0.4884(0.5019)0.4884(0.5019)
Depth-0.6630(0.6583)0.6630(0.6583)0.6640(0.6762)0.6640(0.6762)0.4645(0.4913)0.4645(0.4913)0.4781(0.5099)0.4781(0.5099)

Reference is the empty-action video. DINO is the mean aligned DINOv1 B/16 cosine over 81 frames, and LPIPS uses frames {0,20,40,60,80}\{0,20,40,60,80\}. In the camera directions, where specificity reaches near-complete effects (Table 1), specificity alters reference content less than the last six layers for tilt and zoom on both DINO and LPIPS, and the two are similar for pan. Effective layer sets therefore differ in how much content they move, and specificity selects sets that move less.

Restricting the exchange. Within selected heads, exchanging only the value operand under the recipient's routing retains the effect (ρ=0.974\rho=0.974), whereas exchanging query–key routing does not (0.0010.001, Table 6).

In the revised paper, we will move these reference comparisons from App. E into the main text and report effect together with content change.

AuthorsOpenReview ↗

Official Comment for Reviewer H8zp (3/4)

W2 + Q3. Generality and paired LTX uncertainty

We thank the reviewer for this careful assessment and for requesting paired uncertainty, which we now report.

Wan2.2 object motion. Object effects are partial for lateral+ and depth+, but the specificity advantage holds there as well. In sufficiency, specificity exceeds EAP-IG in all four object directions (median ρ\rho 0.340.34 vs −0.32-0.32 for lateral+, 0.650.65 vs −0.45-0.45 for depth+, 0.980.98 vs −0.00-0.00 for lateral−, 0.990.99 vs 0.260.26 for depth−, Table 1). Camera-compensated CoTracker3, which is not used for selection, gives the same ordering, with sign-correctness of 91.2%91.2\% vs 63.8%63.8\% in the negative-direction sufficiency cells (N=80N=80) and 82.5%82.5\% vs 68.1%68.1\% across object necessity (N=160N=160). The step and budget choices are reported with their sweeps in App. F.

LTX selector. LTX ranks individual nodes rather than layers. At the node level, a pure ratio can favor nodes whose attribution is small in every direction. LTX therefore uses Md(j)/Dd(j)M_d(j)/\sqrt{D_d(j)}, the geometric mean of target magnitude and specificity, with no fitted weight. Both selectors share the same 840 candidates and the same 36-node budget.

Q3. Is the advantage statistically supported on LTX? Yes. Paired 95% intervals exclude zero in favor of the specificity-based selector for Tilt+ sufficiency, Zoom+ in both exchanges and Zoom− necessity. Entries are median paired MB-minus-Raw normalized-flow differences (discovery N=50N=50, attribution seed 1).

Direction; evaluation N=40N=40SufficiencyNecessity
Pan+−0.0062[−0.0154,0.0017]-0.0062[-0.0154,0.0017]−0.0036[−0.0151,0.0024]-0.0036[-0.0151,0.0024]
Pan-−0.0058[−0.0271,0.0058]-0.0058[-0.0271,0.0058]−0.0170[−0.0265,−0.0084]-0.0170[-0.0265,-0.0084]
Tilt+0.1645[0.1010,0.2520]0.1645[0.1010,0.2520]0.0120[−0.0006,0.0411]0.0120[-0.0006,0.0411]
Tilt-0.0354[−0.0017,0.1147]0.0354[-0.0017,0.1147]0.0025[−0.0280,0.0424]0.0025[-0.0280,0.0424]
Zoom+0.0394[0.0064,0.0915]0.0394[0.0064,0.0915]0.0154[0.0058,0.0748]0.0154[0.0058,0.0748]
Zoom-0.0173[−0.0012,0.0673]0.0173[-0.0012,0.0673]0.0151[0.0029,0.0468]0.0151[0.0029,0.0468]

The Zoom+ gain is positive in both exchanges across all five archived attribution seeds. Intervals resample matched scenes with the selected sets fixed.

Repeated discovery. We repeated attribution with two disjoint sets of twenty discovery scenes. Pan+, which shows no significant difference in the original setting, shows a significant gain on one set (sufficiency 0.05430.0543 [0.0290,0.1090][0.0290, 0.1090], necessity 0.03870.0387 [0.0283,0.0813][0.0283, 0.0813]), while the other set's intervals include zero. Both repetitions use the same forty evaluation pairs. Selection is also stable to the discovery sample. With calibration fixed, omitting one discovery scene retains 35.6535.65 of 3636 nodes on average across six directions.

In the revised paper, we will add these intervals to Section 4.7, state that LTX supports the advantage for tilt and zoom, and make clear in the abstract that the reported numbers are from Wan2.2.

Q4. Excluding the opposite direction

We thank the reviewer for this suggestion. Excluding the opposite direction leaves the selection and the effect essentially unchanged.

Opposite-exclusion checkResult
Retained layers; ten directions52/6052/60
Changed Pan+ layer27→027\to0
Modified Pan+ ρ\rho; N=10N=10, sufficiency/necessity1.0032/1.00461.0032/1.0046
Sign agreement; both exchanges10/1010/10
Paired change versus original; sufficiency/necessity−0.0017/−0.0067-0.0017/-0.0067

Across the ten directions, 52 of the 60 selected layers are retained. For Pan+, only one layer changes (27 to 0), and the modified set gives the same effect as the original set, with paired changes below 0.010.01 and sign agreement in every pair.

The opposite direction's mass need not mirror the target's. Although Yaˉ=−YaY_{\bar a}=-Y_a, Eaˉ(ℓ)E_{\bar a}(\ell) is computed from the opposite instruction's own prompt pair, so its activation difference and attribution mass differ from those of the target. Including it in the denominator penalizes layers that are attributed to the whole axis regardless of sign, which favors direction-specific layers.

In the revised paper, we will report this opposite-excluded variant in the appendix.

Reviewer_S5RgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes CAP-MED, a way to find which internal parts of a video diffusion transformer (Wan2.2) are responsible for a requested motion such as a camera pan. It generates paired videos from the same scene and seed, with and without a motion instruction, scores nodes (step × layer × submodule) with EAP-IG attribution, and keeps the layers whose attribution for the target motion is high relative to the other motion directions. The selected 36-node set is then tested by swapping activations between the paired runs: inserting instructed activations into the uninstructed run (sufficiency) and the reverse (necessity). The selected sets recover and suppress the motion far more than sets chosen by attribution magnitude alone, and the workflow is repeated on LTX-Video and a color-temperature attribute.

Critical strengths
  • The method is intriguing and the paper is doing something novel.

  • It is nice to understand what each node does inside a video diffusion model.

Critical weaknesses
  • The paper could be better written. It is dense, and key details of the method are hard to find or missing (see questions).

  • Experiments. Understanding what each node does is useful only if it is general. If the selection is highly sensitive to the video distribution, it is more an optimization that happens after the fact than science, because I cannot know beforehand what to change. Explainability is useful if I can also exploit it: such an analysis can be done on any function — find what part changes the output the most and you will always find something. But if it is consistent over many points, I can manipulate that variable. The experiments don't yet show this consistency (questions 4–8), and I'm not sure whether five integration points with no description of the curve IG is applied on are enough.

Questions for authors
  1. Motion score. Y_a = l_a − l_ā (Eq. 1) only compares a direction with its opposite and ignores the other classes (static, other axes), so the score can move even when both p_a and p_ā are tiny. It also doesn't capture the interplay between movements: if I know a node pans right, I don't know what it does to, say, the camera tilting up, or whether the two are correlated. Table 3 (STOR) gives an average off-axis ratio for the whole set, but not which other motions change or how they relate. Why this score, and how often does the tiny-probability case happen on the patched videos?

  2. How the attribution is actually computed. Sec. 3.2 says EAP-IG with a "paired-activation interpolation", but I couldn't find (a) what is interpolated: one node at a time, all nodes at a step jointly, or the inputs; (b) how the gradient of Y_a gets back to step t: regenerating from step t through the VAE and classifier at every integration point, or a one-step x0 estimate; (c) whether code will be released. Without this the selection stage is hard to reproduce.

  3. Five integration points. K_IG = 5 (l. 246) is not justified. If the response changes sharply between two sample points, five points can miss most of it. A check with more points, or comparing the attribution estimate against actual patching for a few nodes, would help. Which α values are used is also not stated.

  4. Independence assumption. Each node's score is its share of a combined swap, and later nodes already carry the effect of earlier ones. Picking 36 of ~1,200 nodes and expecting them to reproduce the effect assumes these contributions roughly add up. We need a way to establish this independence before selection; Table 5 actually shows strong dependence between sets.

  5. Sensitivity to the data distribution. How do I know the selection is not sensitive to the data distribution? For Wan2.2 I couldn't find whether the 40 evaluation pairs are disjoint from the 10 attribution pairs (stated only for LTX), how many distinct scenes were used, or where the scene sentences come from. Each direction also uses one fixed instruction sentence (Table 11). Does the selection hold with paraphrased instructions or a different prompt distribution?

  6. Camera-motion readout. I'm not convinced by how camera movement is measured. The main numbers come from the authors' own VideoMAE classifier trained on Wan2.2 videos; VBench CameraPredict is only a secondary check (Table 4), and the classifier's static labels come from VBench itself (B.1). The same classifier also picks the nodes through its gradients, so nodes that move its logits without real motion would look good on both. There is no human evaluation of the patched videos.

  7. Stability of the selected nodes. Donor and base activations depend on which videos are used. If I take a different mix, say videos a, b, c versus b, c, d for the same movement, does the selection create new nodes, i.e. does the node set for b and c change depending on whether a or d is included? In practice, when I pick a video, how do I know which nodes to change, given that changing nodes could affect other movements too? Every patch also uses the donor or base value directly (α = 1 or 0) from a run of the same scene, so it's not clear how the found nodes would be used for steering, or whether moving the other way (h0 − α·dh) reverses the motion.

  8. Quality of the source videos. The analysis assumes the instructed videos actually show the requested motion. It could be that for videos a, b, c with "pan right", only b actually pans and a and c don't. The paper's own annotation shows prompts often aren't followed (18.1% of object-prompted and 3.2% of camera-prompted videos relabeled, Table 14), and A.5 mentions "pair screening" without saying the criterion for Wan2.2. Since Eq. 2 averages attribution over only 10 pairs, how do we know one video isn't controlling the results? A per-pair breakdown, or a leave-one-pair-out check, would answer this.

4 comments
AuthorsOpenReview ↗

Official Comment for Reviewer S5Rg (1/4)

We thank the reviewer for finding the method intriguing and novel, and for noting the value of understanding what internal nodes do in a video diffusion model. The detailed questions on attribution, stability and motion readouts were very helpful. Following them, we clarify the implemented attribution procedure and report new checks on integration points, pair omission, instruction variants, absolute probabilities and cross-axis changes. We address each question below and hope that these responses resolve the reviewer's concerns.

W1 + Q2 + Q3. Implemented approximation, reproducibility and integration points

We thank the reviewer for these precise questions. Eq. (2) is implemented as written. The paired endpoint activation difference is combined with the gradient signal averaged over KIGK_{\mathrm{IG}} points, and absolute values are taken after averaging over pairs, as in Eq. (3). This procedure produced all reported Wan2.2 selections for the ten directions. The paper left two choices unstated, which we specify here.

(a) Interpolation path. We interpolate the UMT5 text-encoder input embeddings between the two endpoint prompts, Cα=UMT5(E0+α(E1−E0))C_\alpha=\mathrm{UMT5}(E_0+\alpha(E_1-E_0)), at the fixed instructed-trajectory latent. All nodes at a step are therefore evaluated jointly along one path, and gradients are computed at each interpolated conditioning. Prompts use common padding and the union mask, trimmed to the effective length.

(b) Gradient signal. Neither full regeneration nor a one-step x0x_0 estimate is used. At each scored step tt, we use the local objective qa,t=mean(da,t⊙vθ)q_{a,t}=\mathrm{mean}(d_{a,t}\odot v_\theta), where da,t=Cov(vt,Ya)/∥Cov(vt,Ya)∥2d_{a,t}=\mathrm{Cov}(v_t,Y_a)/\|\mathrm{Cov}(v_t,Y_a)\|_2 is the frozen direction along which the step-tt velocity covaries with YaY_a. It is calibrated once from twenty rollouts of the ten discovery pairs. Gradients therefore do not pass through later sampler steps, the VAE or the classifier. This keeps attribution tractable over forty layers and ten steps, and the classifier's gradients do not directly drive selection.

freeze d[t] from calibration
for discovery pair i, scored step t:
  z = instructed latent
  H0,H1 = endpoint-text source fields at z
  delta = H0-H1
  for j=1..5:
    C = UMT5(E0+(j/6)*(E1-E0), union_mask)
    G[j] = grad_destination_fields(mean(d[t]*DiT(z,C,t)))
  A[i,e,t] = dot(delta[source(e)],mean_j G[j][dest(e)])
E[a,l] = sum_{t,e from l} abs(mean_i A[i,e,t])
rank E[a,l]/(mean_other E[b,l]+1e-9); freeze six layers

Source hooks precede the residual gates and destination hooks follow normalization. Attribution scores steps 0,4,…,360,4,\ldots,36 of a forty-step schedule, and interventions exchange SA/FFN fields at steps {0,2,4}\{0,2,4\} of a ten-step schedule.

Integration points and α\alpha. We use α=k/6\alpha=k/6 for k=1,…,5k=1,\ldots,5. To test whether five points suffice, we recomputed the Pan+ attribution with eleven points (α=k/12\alpha=k/12).

Five versus eleven pointsSpearman rank correlationSelected layers retained
Each of the ten pairs, first scored step0.99490.9949 to 0.99980.9998
Ten-pair aggregate with that step recomputed0.99960.99966/66/6
Pair 0, all ten scored steps0.99940.99946/66/6

The ranking is essentially unchanged and the selected layers are identical, so five points suffice for the ranking used in selection. Attribution only proposes candidate sites. Every effect we report is measured by actually patching the selected set into generated videos, so attribution error can affect which sites are proposed but not the validity of the reported effects.

Writing and code. In the revised paper, we will add this algorithm box and the two specified choices to Section 3.2 and list all attribution settings in one appendix table. Code will be released upon acceptance.

AuthorsOpenReview ↗

Official Comment for Reviewer S5Rg (2/4)

W2 + Q5. Frozen selection, disjoint evaluation and instruction variants

We thank the reviewer for raising this concern. CAP-MED selects reusable locations before evaluation. Layers are selected on discovery scenes and frozen, so the reported effects are not fitted to the evaluation videos.

The ten discovery scenes and forty evaluation scenes are disjoint, and their seeds are disjoint. The forty evaluation scenes differ across directions. Selection is also stable when individual discovery pairs are omitted (Q7).

Scene source. Scene sentences are the first sentences of OpenVid-1M captions. We kept 1,000 sentences that name a clearly movable main object under a predefined criterion and crossed each with the ten predefined motion instructions, four for object motion and six for camera motion.

Frozen layers also work with two instruction variants on the same ten original-paper pairs. Both specify stationary subjects. Each wording uses its own endpoint contrast.

Pan+ wording, N=10N=10Classifier ρ\rho, sufficiency/necessityIndependent flow
Smooth rightward pan1.0115/0.97431.0115/0.97430.9905/0.99410.9905/0.9941
Slow continuous rightward pan0.9743/1.01610.9743/1.01611.0189/1.00111.0189/1.0011

Signs agree in every tested pair. This supports reuse beyond one literal wording within the scene bank.

In the revised paper, we will state the disjoint split and the scene source in Section 4.

Q1. Directional score, absolute probabilities and cross-axis changes

We thank the reviewer for this careful question.

Why this score. Each motion axis has two opposite directions, so Ya=la−laˉ=log⁡(pa/paˉ)Y_a=l_a-l_{\bar a}=\log(p_a/p_{\bar a}) directly measures the signed direction along that axis. It cancels the shared softmax normalizer, so it does not change when probability mass moves between unrelated classes. The intervention ratio ρ\rho then normalizes each change by the observed instructed-minus-empty contrast of the same pair.

How often both probabilities are tiny. Tiny means max⁡(pa,paˉ)<0.01\max(p_a,p_{\bar a})<0.01.

Original N=40N=40SufficiencyNecessity
Tiny outputs2/402/4034/4034/40
Argmax matches intended endpoint40/4040/4036/4036/40

In sufficiency, where the patched video should pan, the tiny case occurs in only two of forty outputs. In necessity, the patched video should match the empty-action video, which has no pan, so small pan probabilities are the intended outcome. In both conditions, the argmax agrees with the intended endpoint in nearly all outputs.

Effects on other motions. Raw score changes before donor subtraction are:

ConditionPanTiltZoomLateralDepth
Endpoint contrast13.65413.654−0.915-0.915−3.224-3.2242.5482.548−0.886-0.886
Sufficiency13.62513.625−1.003-1.003−3.419-3.4191.7301.730−0.696-0.696
Necessity13.58613.586−0.740-0.740−3.958-3.9582.9842.984−1.855-1.855

On every other axis, the changes have the same sign as the endpoint contrast and a similar magnitude. The intervention therefore reproduces the instruction's own side effects rather than adding unrelated motion. Across pairs, pan changes correlate only weakly to moderately with tilt (−0.348-0.348 and −0.271-0.271 in sufficiency and necessity) and zoom (0.2110.211 and 0.1100.110).

In the revised paper, we will add this per-axis response table to the appendix alongside STOR.

AuthorsOpenReview ↗

Official Comment for Reviewer S5Rg (3/4)

Q4. Dependence between intervention sites

We thank the reviewer for this point. We agree that contributions need not add up, and our procedure does not assume that they do.

No additivity is assumed. Attribution only ranks candidate sites. Scores are never summed to predict a video effect. Instead, the selected set is exchanged jointly, and its effect is measured directly on forty held-out scenes. Whether the selected set reproduces the effect is therefore tested empirically rather than inferred from the scores, and it does so with aggregate median ρ\rho of 0.990.99 in sufficiency and 0.970.97 in necessity (Table 1).

Carry-over across steps. Each scored step is attributed locally at a fixed latent, and gradients do not pass through later sampler steps (Q2). A later step's score therefore does not absorb effects propagated from earlier steps through the sampler. Within a step, attribution is assigned to edges between submodules, so an effect is credited to the edge that carries it rather than repeated at every later layer.

Dependence is a result, not a contradiction. In the separate fifteen-pair head test, the later empty-state clamp reduces the earlier Pan+ effect from 0.98170.9817 to 0.01140.0114, and the same pattern holds in all eight cells of Table 5 across four motion families (median changes of 96.596.5 and 95.795.7 pp in the two exchange directions). This is exactly why we evaluate whole sets jointly instead of predicting their effect from individual scores.

In the revised paper, we will state explicitly in Section 3 that selection does not rely on additivity and that every reported effect comes from joint exchange.

Q6. Classifier-independent motion measurement

We thank the reviewer for raising this concern. Two features of our design prevent nodes that only move the classifier's logits from appearing successful.

The classifier's gradients are not used for selection. As described in Q2, attribution differentiates a local objective at each denoising step. Gradients never pass through the VAE or the classifier. The classifier enters only through a fixed direction calibrated from rollouts, so selection cannot exploit the classifier's gradients.

Readouts that are not used for selection give the same ordering. Optical flow and point tracking measure pixel and point displacement directly, without a learned motion classifier. A node that changes logits without producing motion would therefore fail these readouts.

Readout not used for selectionScopeCAP-MEDEAP-IG top-36
Dense optical flow, sign-correctnessSufficiency, all ten directions, N=400N=400 per selector80.7%80.7\% [77.0,84.5][77.0, 84.5]62.7%62.7\% [58.0,67.5][58.0, 67.5]
CoTracker3, sign-correctnessObject necessity, N=160N=160 per selector82.5%82.5\%68.1%68.1\%
VBench CameraPredict, successTwelve camera comparisonsHigher in 12/1212/12

The new identical-grid comparison shows the same pattern under independent flow on the same forty Pan+ pairs and endpoints.

ExchangeSPEC raw effectMagnitudePaired gain
Sufficiency4.45824.45820.00180.00183.72183.7218
Necessity4.23454.23450.18370.18374.50054.5005

Patched flow also follows the time course of its own donor (median correlation 0.9550.955 for pan) rather than other sources of the same direction (0.0200.020), which a logit-only change would not produce. Qualitative examples are shown in Figures 3 to 8.

In the revised paper, we will summarize these selection-independent readouts in the main text and state that the classifier is not differentiated during selection.

AuthorsOpenReview ↗

Official Comment for Reviewer S5Rg (4/4)

Q7 + Q8. Discovery stability, source contrasts and practical use

We thank the reviewer for these questions on stability and use.

No single discovery video controls the selection (Q7, Q8). We recomputed the Pan+ aggregate while omitting each of the ten discovery pairs in turn.

CheckResult
Mean retained layers5.2/65.2/6
Omissions retaining at least five9/109/10
Always retained23,37,39
Positive source margin/contrast9/109/10 each
Median contrast13.219613.2196
∣TE∣≤10−3\lvert TE\rvert\leq10^{-3}0/100/10

On average, 5.2 of the six layers are retained, at least five are retained in nine of ten omissions, and layers 23, 37 and 39 are retained in every omission. Nine of the ten discovery pairs show a positive instructed-minus-empty contrast with median 13.2213.22, and no pair has a near-zero instruction effect. The calibration direction is kept fixed across omissions.

Source quality and screening (Q8). Evaluation pairs are screened by ∣Ya(V0)∣<1\lvert Y_a(V_0)\rvert<1, which excludes pairs whose empty-action video already shows the target motion. Because ρ\rho normalizes each patched change by the pair's own observed contrast, it measures how much of the instruction's actual effect is transferred, even when that effect is weaker than intended. The readouts in Q6, which measure actual displacement rather than instruction labels, confirm the same ordering across all ten directions.

Which sites to use (Q7). For a given direction, one uses that direction's frozen set without a new search. Specificity already penalizes layers that are attributed to other directions, and the changes on other axes follow those produced by the instruction itself (Q1).

Steering and reversal (Q7). We use paired-state exchange with α∈{0,1}\alpha\in\{0,1\}. Exchanging toward the empty-action state removes the motion (necessity). The opposite motion is obtained with the opposite direction's own frozen set, for example Pan− with median ρ=1.00\rho=1.00, rather than by negating the difference, which did not give consistent reversal.

In the revised paper, we will add the per-pair omission analysis and the screening criterion to the appendix.

Reviewer_NEPQOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper ranks internal locations in video diffusion Transformers using target-versus-other motion attribution. It then exchanges activations between motion-instructed and uninstructed generations to test whether the selected locations affect motion. The main experiments are on Wan2.2, with additional analyses on LTX-Video.

Critical strengths
  • The authors test whether swapping the selected activations makes the target motion stronger or weaker. They also measure motion in the resulting videos using optical flow and tracking, rather than relying only on the classifier score.
  • The Wan2.2 results show large intervention effects at the reported node budget.
Critical weaknesses
  1. The paper does not clearly establish why identifying these large activation fields is an important scientific problem. The motivation explains why attribution alone is insufficient, but the results do not yield a clear account of how the model computes motion or a demonstrated use for the identified sets.
  2. The central comparison does not isolate the proposed specificity criterion. The proposed method uses a fixed step–submodule grid, while the raw-attribution baseline can allocate its nodes differently. Given the paper’s own evidence that intervention location matters greatly, a top-attribution baseline on the identical grid is essential.
  3. Each “node” is an entire token-by-channel activation field. Swapping 36 such fields may transfer broad information about the generated video, not just motion. The experiments show that the swaps affect motion, but do not convincingly establish motion-specific internal structure.
  4. The presentation moves between edge attribution, layer selection, full-field patching, and separate head-level analyses without making their relationship to one central claim clear. The LTX-Video experiment also changes the selector and shows much weaker, nonuniform gains.
Questions for authors
  • What concrete insight into motion generation follows from finding these fields, beyond showing that swapping them changes the measured output?
  • Can finer-grained interventions distinguish motion transfer from broader transfer of video content?
3 comments
AuthorsOpenReview ↗

Official Comment for Reviewer NEPQ (1/2)

We thank the reviewer for the careful review and for recognizing that we test whether the selected activations strengthen or weaken the target motion, that we measure the resulting motion with optical flow and tracking rather than relying only on the classifier score, and that the Wan2.2 interventions produce large effects. Following the feedback, we added an identical-grid comparison with attribution-magnitude ranking and clarified how content transfer relates to the instructed reference. We address each concern below and hope that these responses resolve them.

W1 + Q1. Scientific contribution and concrete insight

We thank the reviewer for this question.

Motivation. Attribution is widely used to localize where a model processes an input. In iterative denoising, however, an effect introduced at a site may not persist to the output (Table R1, rows 1 and 3). Identifying where an instruction's effect is transmitted therefore requires selection validated by intervention.

Findings on motion generation. Each test varies one factor at a fixed intervention budget (Table R1).

FindingControlled testResult
Timing determines transmission12 fields, step 0 vs step 2 (Pan+, N=40N=40)ρ\rho 0.030.03 vs 0.990.99
The effect is carried by attention values300 head-steps, QK vs V exchange (N=10N=10)ρ\rho 0.0010.001 vs 0.9740.974
Later computation can override the effectTwo head sets, four motion families, both exchange directions8/8 cells, median change 96.596.5 and 95.795.7 pp
Transfer is source-specificPatched flow time course vs own source and other same-direction sourcesMedian correlation 0.9550.955 vs 0.0200.020

These results extend beyond the observation that exchanges alter the output. Equal-sized exchanges produce either no effect or complete transfer depending on timing. Under the recipient's own routing, the instruction acts primarily through the values that attention writes. An effect introduced at one set persists only when a later set receives the matching state, which indicates a two-stage conditional dependence. Patched videos follow the time course of their own donor, so the exchanged sites do not function as a fixed directional switch.

Use. Sets identified on ten discovery scenes are frozen and reused on forty disjoint evaluation scenes without further search. They provide fixed locations for controlled tests of where an instruction's effect enters and whether it persists. In the revised paper, we will state this motivation in the introduction and present these findings together in one section.

W2. Identical-grid target-attribution comparison

We thank the reviewer for this important suggestion. Following it, we compared specificity with target-magnitude ranking on the identical grid. On this grid, specificity identifies effective sites where attribution magnitude does not.

The two selectors differ only in the ranking rule. Both use the same target-attribution score Ea(ℓ)E_a(\ell), the same six-layer budget, steps {0,2,4}\{0,2,4\}, SA/FFN submodules and the same forty Pan+ evaluation pairs. Magnitude ranks layers by Ea(ℓ)E_a(\ell), whereas specificity divides it by the mean attribution of the other directions. The comparison therefore isolates the specificity criterion from step and submodule allocation. We rank at the layer level, the same unit used by specificity.

Selector (Pan+, N=40N=40)Sufficiency ρ\rhoNecessity ρ\rhoSign agreement (suff., nec.)Independent flow effect (suff., nec.)
Specificity1.00111.00110.98930.989340/4040/40, 40/4040/404.464.46, 4.234.23
Same-grid magnitude−0.0169-0.01690.02430.024318/4018/40, 27/4027/400.000.00, 0.180.18

Values are medians. Sufficiency inserts instructed activations into the empty-action run, and necessity exchanges in the opposite direction. The classifier and the independent flow readout give the same ordering.

In the revised paper, we will add this identical-grid comparison to the main results and use it as the primary test of the specificity criterion. We thank the reviewer again for suggesting this control, which makes the central comparison considerably clearer.

AuthorsOpenReview ↗

Official Comment for Reviewer NEPQ (2/2)

W3 + Q2. Intervention granularity and transfer of video content

We thank the reviewer for raising this point. Our paired design and reference comparisons directly constrain what a full-field exchange can transfer.

What can be transferred is limited to the instruction's effect. Each pair shares the scene prompt, seed and sampler and differs only in the appended instruction. An exchange therefore cannot import unrelated scene content. It can transfer only differences caused by the instruction, which include the motion itself and its accompanying changes, such as newly revealed regions under camera motion.

The transferred change tracks the instructed change rather than exceeding it.

Pooled sufficiency (App. E)CAP-MED patchedInstructed videoEAP-IG patched
Subject consistency change−0.038-0.038−0.043-0.043−0.017-0.017
Background consistency change−0.025-0.025−0.027-0.027−0.012-0.012
Aesthetic quality change−0.037-0.037−0.048-0.048−0.018-0.018
FVD to instructed reference328328443443

Changes are relative to the empty-action video. CAP-MED changes the video by about as much as the instruction itself does, and its outputs are closer to the instructed videos than EAP-IG outputs are. EAP-IG changes the video less because it also induces much less motion (median ρ\rho 0.150.15). Patched flow also follows the time course of its own donor (median correlation 0.9550.955) rather than other sources of the same direction (0.0200.020).

A finer intervention localizes the effect within attention. Exchanging only the value operand under the recipient's routing carries almost the full effect, whereas exchanging query–key routing carries almost none (Table R1 in W1).

Interpretation. The selected sites therefore transmit the instruction's effect, and the transferred change stays within what the instruction itself produces. This is why we describe them as motion-effect sites rather than as a motion-only subspace. In the revised paper, we will state this interpretation in the abstract and Section 4.5 and move the reference comparison from App. E into the main text. We thank the reviewer for this question, which sharpened the interpretation of our results.

W4. Analysis hierarchy and the LTX selector

We thank the reviewer for this comment and agree that the relationship between the analyses should be clearer. The central claim is that target-versus-other specificity identifies sites whose exchange transmits a motion instruction's effect. Each analysis has one role in testing this claim.

AnalysisUnitRole
Edge attributionEdges between SA/FFN outputsScores candidate sites
Specificity selectionLayers on a fixed step–submodule gridSelects the site set from discovery scenes only
Full-field exchange36 SA/FFN fieldsTests the frozen set on disjoint evaluation scenes
Head-level exchangeHead-steps in separate setsTests the selector at a finer unit and examines how the effect is carried within attention (W1)
LTX-VideoNodes in a second architectureTests whether the procedure transfers

The head-level analyses use separate sets. The matched K=100K=100 comparison tests the selector at the head level, while the operand and two-set exchanges characterize the computation rather than validate the selected 36 fields. In the revised paper, we will open Section 4 with this table and state the unit, discovery data and evaluation sample of each analysis.

LTX-Video. LTX ranks individual nodes rather than layers. At the node level, a pure ratio can favor nodes whose attribution is small in every direction. LTX therefore uses Md(j)/Dd(j)M_d(j)/\sqrt{D_d(j)}, the geometric mean of target magnitude Md(j)M_d(j) and specificity Md(j)/Dd(j)M_d(j)/D_d(j), with no fitted weight. Both selectors share the same 840 candidates and the same 36-node budget. Paired 95% confidence intervals over forty evaluation pairs support the gain for tilt and zoom (Tilt+ sufficiency, Zoom+ in both exchanges and Zoom− necessity), and the Zoom+ gain is positive in both exchanges across all five archived attribution seeds. The gain is not supported for pan. LTX thus shows that the procedure transfers to a second architecture for tilt and zoom, and we will state this scope clearly in Section 4.7.

Thank you for the detailed response. Although the additional experiments and discussion have addressed some of my concerns, I still find the paper’s motivation unconvincing. I have therefore decided to maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.