acceptodds
Under review as a conference paper at ICLR 2027

Prediction Targets in Diffusion Models: Geometry, Capacity, and Shortcuts

Abstract

Although -, -, and -prediction yield the same score function at optimum, their prediction targets can have markedly different geometric structures. Empirical studies have reported substantial performance gaps between these parameterizations in both pixel and representation spaces. We study this discrepancy through the interaction between prediction-target geometry and limited output rank. For data with low-dimensional structure, the clean predictor follows the underlying signal structure, whereas noise and velocity predictors additionally carry ambient components inherited from the noisy input. These additional components can create ambient approximation floors for - and -prediction, even when the same output rank is sufficient for -prediction. This geometric characterization also suggests a simple time-dependent shortcut that directly carries the ambient component, leaving the core network to represent a scaled conditional clean signal. With the appropriate coefficient, the shortcut removes the resulting ambient approximation floor. Controlled experiments validate the predicted approximation scaling and evaluate the learned shortcut on nonlinear synthetic data. On ImageNet with JiT-B/16, the shortcut reduces -prediction FID from to , compared with for -prediction.

Primary area:
generative models
Keywords:
diffusion models, prediction targets

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_JDxtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper presents a study on the impact of different prediction target, signal (x-), noise (\epsilon-), and velocity (v-) predictions, on the final performance of the diffusion model. The authors explain the performance gaps between these different targets through the interaction between the prediction-target geometry and finite model capacity. They present several theorems for the affine low-dimensional subspace, low-rank Gaussian prior, mixtures of low-rank Gaussians, which show that the clean prediction (x-prediction) remains in a low dimensional subspace but the noise and velocity predictions scale with the ambient dimension D. Based on this geometric characterization, the authors propose a time-dependent shortcut that directly carries the ambient component, leaving the core network to represent the signal. Their empirical evaluation support their theoretical findings.

Critical strengths

In my view, the main strengthen of the paper is the geometric characterization of the impact of different prediction targets. The authors provides a theoretical explanation on why different targets could lead to different performance outcomes, even though they lead to the same score function at optimum.

The shortcut proposal also seems to be a nice contribution that produces large performance improvement in the ImageNet test, without adding significant complexity in the learning algorithm.

Critical weaknesses

One main concern is that all theorems seem to be straightforward, given the assumptions made prior for each theorem. There are not enough discussions on these assumptions, such as how far they are from the practical scenarios, and how we should interpret the theoretical results when these assumptions are violated. My impression is that the theorems provide some nice results but only under the context of the given assumptions, and they may not explain the real performance issues observed in the diffusion models. I believe a more detailed discussions and analysis on these assumptions and the gaps between the assumptions and the real situations are needed.

Another concern is on the empirical evaluation. The experimental evaluation only occupies one page in the main text, and the validation does not seem to be comprehensive enough to support all their findings.

Questions for authors

As already discussed in the weaknesses section, my main concern is on the assumptions that support the theorems. Can these assumptions been weakened, and be made closer to the practical scenarios? The theorems seem to be straightforward given these assumptions, so do these theorems really support what the authors want to claim on the impact of different prediction targets, or they are some simple consequences of the assumptions? What are the technical novelties in the proofs of these theorems on top of the assumptions made?

2 comments
AuthorsOpenReview ↗

Author Response

We thank the reviewer for the constructive feedback. We have uploaded a revised manuscript, with changes highlighted in blue. All figure, table, equation, and section references below refer to this revision.

Weakness 1 [Assumptions and practical relevance]

Spatial, semantic, and temporal redundancy in images and videos motivates our structured signal models, whereas injected Gaussian noise spans the ambient space.

Our analysis extends beyond exact subspaces. Theorem 1 allows arbitrary distributions on an affine subspace, Theorem 3 considers low-rank Gaussian mixtures, and Theorem 6 addresses smooth manifold unions with off-manifold perturbations. These results show how geometric separation can persist for more complex structures under the stated conditions. The target relationships (Eq. 14) hold for arbitrary finite-second-moment data distributions; the separation and approximation advantages depend on the data structure and need not persist when that structure is absent.

Our analysis also provides a theoretical perspective on existing architectural designs. RAE-based models benefit from wider diffusion heads (Zheng et al., 2026), U-Nets use multiscale skip connections (Ho et al., 2020), and EDM has an input-to-output skip (Karras et al., 2022). Our geometric analysis helps explain why capacity and input pathways matter: they affect which target components the core must represent. In particular, the target-matched shortcut handles the ambient component outside the core.

Weakness 2 [Empirical evaluation]

Our experiments test the approximation predictions and examine their relevance to learning. In Figure 1, clean-prediction error becomes negligible once the signal fits, whereas noise/velocity errors grow with ambient dimension and fall with output rank. Figure 2 contrasts a persistent finite-budget gap at D=512D=512 with slower but successful direct-vv learning at D=32D=32. Figure 5 shows improved Gaussian-mixture recovery with shortcuts.

Appendix D.3 (Table 3) further validates the predicted approximation behavior: direct vv-prediction exhibits increasing error with ambient dimension, EDM reduces but does not eliminate this error, while both fixed and learned shortcuts achieve nearly zero approximation error.

New sampling comparisons (Appendix F.3, Figures 6–8, pp.34–36). We compare direct prediction, EDM-style preconditioning, and fixed/learned shortcuts for all three targets. At D=512D=512, both shortcuts recover clearer projected spirals than direct ε/v\varepsilon/v-prediction.

New coefficient analysis (Figure 3, p.10) gives mean relative coefficient errors of 4.32% (ImageNet) and 0.81% (Swiss roll) on the uniform evaluation grid for t<0.95t<0.95 against −1/d(t)-1/d(t), d(t)=max⁡(1−t,0.05)d(t)=\max(1-t,0.05). The agreement supports our interpretation that the input branch takes on the ambient component predicted by the theory (Appendices I.2/I.5).

New vv-EDM comparison (Table 2, p.9). We adapt EDM preconditioning to vv-prediction (Appendix D.2) under the same JiT-B/16 configuration (200 epochs). ImageNet 256×256256\times256 results are:

MethodFID ↓
xx-prediction with vv-loss8.62
Direct vv96.53
vv-EDM (new)60.30
Learned shortcut-vv7.83

These results support our architectural interpretation: explicitly handling the input-dependent component can substantially improve vv-prediction performance.

Questions for Authors [Theoretical significance and technical contributions]

The relaxation of assumptions is addressed in Weakness 1. Our contribution goes beyond geometric differences by quantifying their approximation consequences. Theorem 2 optimizes each target's output representation independently and shows that approximation gaps can persist after optimization error vanishes. Theorem 5 establishes how a target-matched shortcut removes the corresponding Gaussian approximation floor. Thus, score-equivalent targets need not place equivalent demands on the core.

The proofs derive target-specific spectral bounds and combine them with established convergence results (Theorem 2), prove Jacobian nondegeneracy for local nonlinear Gaussian-mixture learning (Theorem 4), and establish normal-energy bounds near smooth manifold unions (Theorem 6). These steps address approximation and learning, rather than only restating the assumed data geometry.

References

  • Zheng et al. (2026). Diffusion Transformers with Representation Autoencoders. ICLR.
  • Ho et al. (2020). Denoising Diffusion Probabilistic Models. NeurIPS.
  • Karras et al. (2022). Elucidating the Design Space of Diffusion-Based Generative Models. NeurIPS.
  • Li & He (2026). Back to Basics: Let Denoising Generative Models Denoise. CVPR.

Thank you again for your time and thoughtful feedback. As the author–reviewer discussion period draws to a close, we would like to ask whether our responses have addressed your concerns and whether you have any further questions we could clarify.

Reviewer_5ELrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks why xx-, ε\varepsilon- and vv-prediction behave differently in practice, even though they share the same optimal score. Its answer combines the geometry of each prediction target with a limit on the model's output rank. When data lie in a low-dimensional subspace, the optimal clean-data predictor stays in that subspace, whereas the optimal noise and velocity predictors keep a full-dimensional component that is a fixed multiple of the noisy input. If the network's output rank is smaller than the ambient dimension, this component cannot be represented, so noise and velocity prediction incur an approximation error that grows with the ambient dimension, while clean-data prediction does not. The paper proves this for low-rank Gaussian data with linear models trained by gradient descent, extends the geometric picture to manifold unions, Gaussian mixtures and Mean Flow, and gives a local convergence result for a simple nonlinear model. Based on this decomposition, it proposes a target-matched shortcut, which adds a learned scalar multiple of the noisy input to the network output. With the theoretically derived coefficient this removes the approximation error, whereas EDM's skip coefficient only reduces it. Experiments on Gaussian data, a Swiss roll and Gaussian mixtures support the predicted behavior. On ImageNet 256 × 256 with JiT-B/16, the shortcut improves vv-prediction FID from 96.53 to 7.83, compared with 8.62 for xx-prediction.

Critical strengths

S1. A clean, general formalization of a widely held intuition. Li & He (2025) explained the failure of noise and velocity prediction informally, through the manifold hypothesis. This paper makes the explanation precise:

  • Theorem 1 holds for any distribution supported on the subspace.
  • Theorem 2 gives upper and lower floors of matching order, both linear in D−mD - m.
  • The separation carries over to manifold unions, mixtures, and Mean Flow.
  • Figure 1 confirms the predicted scaling quantitatively.

This gives the community a precise statement to cite and build on.

S2. Unifying identities that lead to a principled design rule. Eq. 14 writes each of ε⋆\varepsilon^\star and v⋆v^\star as a known multiple of xtx_t plus a multiple of x⋆x^\star. This explains why the targets behave differently. It also fixes the skip coefficient γτ(t)\gamma_\tau(t) from the target itself rather than by heuristic tuning. Appendix D.2 and Table 2 show that EDM's cskipc_{\text{skip}} still leaves an ambient floor, while the derived coefficient removes it. The learned coefficient also converges to the theoretical value. This is a useful lens for designing preconditioning.

S3. Clear exposition and well-controlled experiments. The experiments progress from quantitative Gaussian checks, through the Swiss roll with and without a bottleneck, to mixtures and ImageNet, and the progression is easy to follow. Within each seed, the methods share data, initialization and training noise. The limitations are stated candidly in Section 6, including that the benefit seen without a bottleneck is not yet explained.

Critical weaknesses

W1. The theory covers the bottleneck regime, but not the ImageNet failure it is used to motivate. The paper asks why the parameterizations differ in practice, and its answer is an approximation floor caused by limited output rank. But Appendix I.5 states that in JiT-B/16 the hidden width and the per-patch output dimension are both 768, so "the output-head dimensions do not impose a rank bottleneck." Yet direct vv-prediction still reaches only 96.53 FID. The same holds for the Swiss roll at D=32D = 32. Section 6 concedes that "the mechanism underlying this benefit remains to be theoretically characterized."

Theorems 2–5 therefore apply to an idealized bottleneck that the practical model does not have, so they do not by themselves explain the ImageNet gap. Figure 2b, where vv-prediction learns more slowly but eventually succeeds, points to a learning or conditioning effect rather than a representability floor.

W2. At its prescribed coefficient, the shortcut coincides with xx-prediction, which limits the novelty of the method. Under the paper's linear path xt=tx0+(1−t)εx_t = t x_0 + (1 - t) \varepsilon (Appendix I.2), bt=1−tb_t = 1 - t, bt′=−1b_t' = -1 and Δt=−1\Delta_t = -1. Hence

γv(t)=−11−t,v⋆=x⋆−xt1−t.\gamma_v(t) = -\frac{1}{1 - t}, \qquad v^\star = \frac{x^\star - x_t}{1 - t}.

At its prescribed coefficient, the shortcut is therefore exactly the x→vx \to v conversion the paper uses for its own xx-prediction baseline, which it describes as "converting xx-predictions to velocity with denominators clamped below at 0.05" (Appendix I.2). The two differ only in that cc is learned and the core output is not divided by 1−t1 - t. It is therefore expected by construction that vv-prediction with the shortcut behaves like xx-prediction. The 96.53→7.8396.53 \to 7.83 gain is therefore best read as reproducing the known gap between xx- and vv-prediction (Li & He, 2025), rather than as evidence for a new remedy.

Questions for authors
  1. Shortcut vs. xx-prediction. Under your path, γv(t)=−1/(1−t)\gamma_v(t) = -1/(1 - t), so the prescribed shortcut equals the x→vx \to v conversion used for the xx-prediction baseline. What exactly differs between the "xx-prediction" and "shortcut-assisted vv-prediction" rows of Table 1: the loss, the clamping, the output scaling, or the parameterization of cϕc_\phi?

  2. Mechanism without a bottleneck. JiT-B/16 has no output-rank bottleneck, so what do you think makes direct vv-prediction fail?

2 comments
AuthorsOpenReview ↗

Author Response

We thank the reviewer for the constructive feedback. We have uploaded a revised manuscript, with changes highlighted in blue. All figure, table, equation, and section references below refer to this revision.

Weakness 1 [Target geometry and learning beyond output-rank bottlenecks]

Our analysis begins with geometric differences between prediction targets, rather than output-rank bottlenecks. Theorem 1 reveals the additional ambient components in direct ε/v\varepsilon/v-prediction, while Theorem 6 extends the geometric analysis to smooth manifold unions under its stated conditions.

Theorems 2–5 quantify the approximation consequences of these differences under output-rank constraints, but the geometric distinction itself is more general. Even without an output-rank bottleneck, learning the additional ambient component may be harder. Figure 2b shows that at D=32D=32, direct vv-prediction recovers the Swiss roll more slowly. We interpret the ImageNet gap as a possible learning difficulty, not a floor established by the rank-constrained theory. Section 5's revised discussion distinguishes these claims.

Weakness 2 [Equivalence to JiT and architectural contribution]

We agree that under the linear path, our analytic shortcut is mathematically equivalent to JiT's xx-prediction with vv-loss under corresponding core scaling. Our contribution is to explain the representational significance: JiT's xx-prediction handles the full-dimensional input-dependent term outside the core, whereas direct vv-prediction must learn it alongside the structured clean signal.

Beyond this mathematical equivalence, we interpret the connection from an architectural perspective. RAE-based diffusion transformers benefit from wider diffusion heads (Zheng et al., 2026), U-Nets employ multiscale skip connections (Ho et al., 2020), and EDM uses an input-to-output skip (Karras et al., 2022). Our analysis offers a perspective on these designs: capacity and information pathways affect how signal and ambient components are handled. The target-matched shortcut explicitly handles the ambient component outside the core. Equation (14) extends this interpretation to general noise schedules, providing guidance for target-specific skip connections and preconditioning.

Question 1 [Shortcut versus xx-prediction]

Both methods use the same velocity loss and sampler, but differ in what the core network learns. JiT's xx-prediction learns the clean signal xθx_\theta, while our shortcut core learns its velocity-scaled component, with cϕ(t)xtc_\phi(t)x_t handling the input-dependent term. During sampling, JiT converts its output to velocity via (xθ−xt)/d(t)(x_\theta-x_t)/d(t), whereas our shortcut directly produces fθ+cϕ(t)xtf_\theta+c_\phi(t)x_t. The two are mathematically equivalent when fθ=xθ/d(t)f_\theta=x_\theta/d(t) and cϕ(t)=−1/d(t)c_\phi(t)=-1/d(t), where d(t)=max⁡(1−t,0.05)d(t)=\max(1-t,0.05).

New coefficient diagnostic (Figure 3, p.10). Relative to −1/d(t)-1/d(t), the mean relative coefficient errors on the uniform evaluation grid for t<0.95t<0.95 are 4.32% (ImageNet) and 0.81% (Swiss roll); see Appendices I.2/I.5. The agreement supports our interpretation that the input branch takes on the ambient component predicted by the theory.

The FIDs of 8.62 (xx-prediction) and 7.83 (learned shortcut) in Table 2 may differ because of coefficient mismatch, core scaling, or optimization; these factors have not been isolated. The experiment demonstrates the effectiveness of an explicit input-to-output connection, consistent with the architectural interpretation provided by our analysis.

Question 2 [Mechanism without an output-rank bottleneck]

Direct vv-prediction must learn both the structured signal and the full-dimensional ambient component, whereas xx-prediction and shortcut-assisted vv-prediction handle the latter outside the core. We expect this additional burden to make learning more difficult without a suitable architectural design, even in the absence of an output-rank bottleneck, potentially leading to much slower convergence. Figure 2b supports this interpretation: at D=32D=32, direct vv-prediction learns more slowly but eventually succeeds. A similar learning difficulty may contribute to its poor ImageNet performance, although a formal characterization remains open.

References

  • Zheng et al. (2026). Diffusion Transformers with Representation Autoencoders. ICLR.
  • Ho et al. (2020). Denoising Diffusion Probabilistic Models. NeurIPS.
  • Karras et al. (2022). Elucidating the Design Space of Diffusion-Based Generative Models. NeurIPS.
  • Li & He (2026). Back to Basics: Let Denoising Generative Models Denoise. CVPR.

Thank you again for your time and thoughtful feedback. As the author–reviewer discussion period draws to a close, we would like to ask whether our responses have addressed your concerns and whether you have any further questions we could clarify.

Reviewer_Gah9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores why x, ϵ\epsilon and v-prediction behave differently in practice despite they share the same optimal score. A prominent example is the failure of ϵ\epsilon and v-prediction in pixel space reported by JiT. And the proposed explanation is the interaction between the geometry of the prediction target and limited output rank. Under a rank-m output constraint, the authors quantify the resulting approximation floors in linear Gaussian models and prove gradient descent convergence for a two-layer linear model and local convergence for a one-gate model on a two-component Gaussian mixture. They propose to add a time-dependent scalar shortcut to the network output and show that the analytic coefficient removes the floor. On the ImageNet with JiT-B/16, the shortcut could reduce the v-prediction FID from 96.53 to 7.83, which is even better compared with the x-prediction.

Critical strengths
  1. This paper explains the x, ϵ\epsilon and v-prediction gap, which is a meaningful practical question.
  2. The proposed modification is very simple.
  3. The assumptions are stated explicitly and the floor bounds for each kind of prediction are quantified cleanly.
Critical weaknesses
  1. The proposed shortcut seems like a different version of the JiT's x-prediction. In JiT, x-prediction under the v-loss is implemented by computing the velocity from the network output as (netθ−zt)/(1−t)(net_\theta − z_t)/(1−t). Under the same linear path, the paper’s ideal coefficient is γv=−1/(1−t)\gamma_v=−1/(1−t), and the core target is x∗/(1−t)x^*/(1−t). So shortcut-assisted v-prediction with the analytic coefficient is similar as JiT’s x-prediction, except that the core output is rescaled by 1/(1−t)1/(1−t).
  2. The experiment part is limited. The whole paper only has one "Shortcut-assisted v-prediction 7.83" result and no ablations. What about a fixed analytic c versus a learned c, and the learned cΦ(t)c_\Phi(t) curve itself. There is no results with the shortcut on larger models such as JiT/32 with 512x512 resolution. There are also no latent or representation-space experiments.
  3. The presentation is a bit hard to follow because of some repeated notations, undefined symbols and organization.
Questions for authors

With c=γvc =\gamma_v, is the shortcut model JiT’s x-prediction scaled by 1/(1−t)? Could you report the runs with the original JiT's x-prediction with its output scaled by 1/(1-t)? Could you plot the learned cϕ(t)c_\phi(t) on ImageNet against −1/(1−t)?

4 comments
AuthorsOpenReview ↗

Author Response

We thank the reviewer for the constructive feedback. We have uploaded a revised manuscript, with changes highlighted in blue. All figure, table, equation, and section references below refer to this revision.

Weakness 1 [Relation to JiT and theoretical contribution]

Our main contribution is to explain and quantify how prediction-target geometry interacts with network capacity and architecture. Although xx-, ε\varepsilon-, and vv-prediction are equivalent at the Bayes optimum, they can impose different representational demands. Theorem 1 characterizes their geometric differences, while Theorems 2–5 analyze approximation, learning, and shortcut design.

This analysis provides an architectural interpretation of prediction targets. Under structured data assumptions, xx-prediction focuses the core on the clean signal, whereas direct ε/v\varepsilon/v-prediction additionally requires modeling ambient components. RAE-based models benefit from wider diffusion heads (Zheng et al., 2026), U-Nets employ multiscale skip connections (Ho et al., 2020), and EDM uses an input-to-output skip (Karras et al., 2022). Our geometric analysis helps explain why these architectural choices matter for different prediction targets: network capacity and information pathways influence how structured signals and ambient components are handled. In particular, the target-matched shortcut explicitly handles the ambient component outside the core, reducing its representational burden.

Under the linear path, our analytic shortcut is mathematically equivalent to JiT's xx-prediction with vv-loss under corresponding core scaling: fθ=xθ/(1−t)f_\theta=x_\theta/(1-t) and c(t)=−1/(1−t)c(t)=-1/(1-t) give v^=(xθ−xt)/(1−t)\hat v=(x_\theta-x_t)/(1-t). Our contribution is to explain the representational significance of this equivalence: JiT's conversion handles the input-dependent term analytically, whereas direct vv-prediction must model it within the core. Our theory quantifies the approximation burden and extends the interpretation to general noise schedules, informing target-specific skip connections and preconditioning.

Weakness 2 [Experimental evaluation and scope]

Fixed versus learned coefficients. Appendix D.3 (Table 3) shows that both fixed and learned shortcuts reduce the Gaussian approximation error to nearly zero, whereas direct vv-prediction and EDM retain nonzero errors. This confirms the benefit of the theoretically predicted coefficient, with comparable results for fixed and learned shortcuts.

New sampling comparisons (Appendix F.3, Figures 6–8) cover direct, EDM-style, and fixed/learned shortcuts for all three targets. At D=512D=512, both shortcuts recover clearer projected spirals than direct ε/v\varepsilon/v-prediction, supporting the benefit of the shortcut beyond the Gaussian check.

New EDM comparison (Table 2, p.9). We adapt EDM preconditioning to vv-prediction (Appendix D.2) under the same JiT-B/16 configuration (200 epochs). ImageNet 256×256256\times256 results are:

MethodFID ↓
xx-prediction with vv-loss8.62
Direct vv96.53
vv-EDM (new)60.30
Learned shortcut-vv7.83

The shortcut achieves performance comparable to xx-prediction, while vv-EDM improves direct vv-prediction but still leaves a substantial performance gap.

Larger-model, 512×512512\times512, and latent/representation-space evaluations are underway, but their computational cost means complete results may not be available during the discussion period. We will report completed results when available.

Weakness 3 [Notation and presentation]

We have added a notation table (Table 1, p.3), clarified symbol definitions, and improved transitions between sections.

Questions for Authors [Equivalence and learned coefficient]

Equivalence. The scaling is already present in JiT's velocity conversion; rewriting the same clean core as an analytic shortcut leaves its predictions unchanged (Table 2, xx-prediction FID 8.62).

Learned coefficient. New Figure 3 shows that the learned coefficients closely match the theoretical prediction, with mean relative errors of 4.32% (ImageNet) and 0.81% (Swiss roll) for t<0.95t<0.95. The errors are measured by ∣1+d(t)cϕ(t)∣|1+d(t)c_\phi(t)|, where d(t)=max⁡(1−t,0.05)d(t)=\max(1-t,0.05) follows Li and He (2026). This agreement supports our interpretation that the shortcut learns to handle the ambient component outside the core network (Appendices I.2/I.5).

References

  • Zheng et al. (2026). Diffusion Transformers with Representation Autoencoders. ICLR.
  • Ho et al. (2020). Denoising Diffusion Probabilistic Models. NeurIPS.
  • Karras et al. (2022). Elucidating the Design Space of Diffusion-Based Generative Models. NeurIPS.
  • Li & He (2026). Back to Basics: Let Denoising Generative Models Denoise. CVPR.

Thank you again for your time and thoughtful feedback. As the author–reviewer discussion period draws to a close, we would like to ask whether our responses have addressed your concerns and whether you have any further questions we could clarify.

Thank you for the response and the revisions. While I appreciate the theoretical analysis of the rank-constrained approximation floors and the new Figure 3, the discussion confirms my main concern: under the standard linear path, the proposed shortcut is mathematically equivalent to JiT's xx-prediction up to a 1/(1−t)1/(1-t) core scaling. Given the incremental novelty over JiT's xx-prediction and the narrow empirical scope, I will maintain my current score.

Thank you for your further feedback. We would like to clarify the focus of our work, which may not have been sufficiently clear in our previous response.

JiT empirically demonstrated that xx-prediction can substantially outperform direct ε/v\varepsilon/v-prediction in its high-dimensional pixel-space setting. Our work investigates when and why such differences arise. We characterize the structured signal and ambient components of prediction targets, extending the geometric analysis from affine subspaces to unions of smooth manifolds. In tractable models, we quantify the resulting capacity requirements: output capacity sufficient for clean prediction can still leave approximation errors that grow with ambient dimension for direct ε/v\varepsilon/v-prediction.

The shortcut follows from this analysis. The target decomposition identifies an explicit noisy-input component that an input-to-output shortcut can handle, leaving the core to represent the signal-dependent part. We further bound the remaining approximation error as the shortcut coefficient varies. Under the linear path, our analytic vv-shortcut is mathematically equivalent to JiT's xx-prediction with vv-loss under corresponding core scaling. We view this equivalence as an instance of the principle revealed by our analysis, which also helps interpret JiT's empirical findings.

This perspective also helps interpret the benefits of wider diffusion heads in RAE-based DiTs and the role of long skip connections in U-Nets in preserving input-dependent information.

More broadly, our theoretical perspective suggests that prediction targets should be considered together with network architecture. In particular, simple shortcuts may help alleviate the difficulties of direct ε/v\varepsilon/v-prediction in high-dimensional pixel-space DiTs.

Regarding empirical scope, we adopt JiT-B/16 as a representative pixel-space DiT architecture for our large-scale evaluation. Our controlled experiments test the approximation predictions and compare learning behavior, while the ImageNet results and coefficient analysis support the relevance of the predicted decomposition in this practical setting. Broader evaluations across architectures, resolutions, and representation spaces would strengthen the empirical evidence.

Thank you again for your thoughtful feedback. We hope this clarification makes the focus and contributions of our work clearer.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We thank all reviewers for their constructive feedback. We have uploaded a revised manuscript with changes highlighted in blue. Below we summarize the main updates; all references are to the revised manuscript.

1. Learned coefficient validation (Figure 3, p.10; Appendices I.2/I.5). The mean relative coefficient errors are 4.32% on ImageNet and 0.81% on Swiss roll, using the same uniform evaluation grid with t<0.95t<0.95. This agreement with the target-matched reference supports our interpretation that the input branch carries the ambient component predicted by the theory.

2. Extended Swiss-roll sampling comparisons (Appendix F.3, Figures 6–8). We compare direct prediction, EDM-style preconditioning, and fixed/learned shortcuts for xx-, ε\varepsilon-, and vv-prediction. Relative to direct ε/v\varepsilon/v-prediction, both fixed and learned shortcuts recover clearer projected spiral structure at D=512D=512 and recover the structure earlier at D=32D=32.

3. New vv-EDM comparison (Table 2, p.9). We evaluate EDM preconditioning adapted to vv-prediction under the same JiT-B/16 configuration and 200-epoch budget. ImageNet 256×256256\times256 FIDs are:

MethodFID ↓
xx-prediction with vv-loss8.62
Direct vv96.53
vv-EDM (new)60.30
Learned shortcut-vv7.83

These results support our architectural interpretation: explicitly handling the input-dependent component can substantially improve vv-prediction performance.

4. Presentation and discussion. We added a notation table (Table 1, p.3), clarified symbol definitions, and improved section transitions and experimental descriptions.

Ongoing evaluations. Evaluations with larger models, higher resolutions, and latent/representation spaces are underway. Given their computational cost, they may extend beyond the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.