- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The manuscript lets different feature channels in a pixel-space diffusion Transformer (JiT) go through different numbers of updates (heterogeneous refinement). The authors find that the channels updated less tend to encode global structure, while the channels updated more tend to encode high-frequency details. Based on this, they propose Persistence Forcing (PerF), which uses the persistent features to modulate the active features, and then builds an extra sampling guidance (Persistence Guidance, PG) by switching this pathway on and off. They report that this consistently beats the JiT baselines on ImageNet 256/512.
Critical strengths
- The motivation and the method connect fairly clearly. The manuscript first observes the feature specialization, then shows that specialization alone is not enough (Table 1), then introduces the conditioning, and finally arrives at PG. The logic is complete.
- The cost-effectiveness of the method is also decent. PerF-L (471M) gets an FID of 1.91, close to the 1.86 of JiT-H (953M). Even after counting the extra forward pass for PG, the compute is still lower than JiT-H.
- The writing and the figures/tables are fairly clear.
Critical weaknesses
-
I think the biggest point of contention in this manuscript is the causal chain from heterogeneous refinement to specialization to performance gain, and also the similarity to Register Guidance. Throughout the Introduction and the Method, the authors keep telling the reader that heterogeneous refinement lets different channels get different refinement budgets, that these channels then spontaneously develop into persistent global features and active local features, and that this improves generation. But the results they actually report come from the combination of heterogeneous refinement only + P-to-A conditioning + Persistence Guidance. This means the current experiments cannot isolate or verify whether heterogeneous refinement is a necessary factor for the final gain. I think the authors should at least run Uniform + conditioning, or Uniform + conditioning + PG, to show whether heterogeneous refinement is really effective. To me, it is entirely possible that the real story is a different one. Maybe the persistent specialization produced by heterogeneous refinement is not what matters, and the authors have simply added an extra conditioning branch by hand, randomly dropped this branch during training, and then used the two predictions for self guidance at sampling time.
-
I think adding REPA to the main table does not mean much. REPA is not a necessary part of the core PerF mechanism. It could be a supplementary result, but it should not stand in for the main PerF results. What readers want to see is how much PerF itself improves things, or what the potential risks are, rather than a result that looks strong but has little meaning. At least for me, it is not a plus.
-
I think the Persistence Guidance component is actually very similar to the Register Guidance proposed in "Registers Matter for Pixel Space Diffusion Transformers". Compared with that method, you train with random P-to-A dropping, while Register Guidance trains with random register dropping. It feels like there is not that much difference. Also, what Figure 3(b) shows as the thing you truly rely on is the existence of an internal conditioning pathway that can be switched on and off. But does this pathway really have to come from the persistent features produced by heterogeneous refinement? The manuscript does not answer this question clearly either (and this is actually a small version of the question in point 1, namely whether heterogeneous refinement is necessary for PG).
-
The FLOPs overhead currently seems to be underestimated. PG + CFG actually needs 3 predictions, but Table 11 only reports the increase in single forward FLOPs. If you compute it, the actual sampling FLOPs come out to about 1.6x the baseline (compared with a normal CFG JiT without PG). That is quite a big gap.
-
Besides that, I also have some suggestions on writing details. For example, the Introduction says "Global structure can often be represented compactly, whereas local textures and fine details require richer representations (Burt & Adelson, 1983; Mallat, 1989; Schwartz & Simoncelli, 2001). ... This motivates us to reconsider whether all hidden features should receive the same amount of refinement in pixel-space DiTs." I feel that "global structure can be represented compactly" leads to the conclusion that representation capacity (width) can differ, while what the paper manipulates is the number of updates (refinement budget). These two are clearly different, and the argument connecting them is missing.
-
The PG training and inference details are currently incomplete. The authors say "We therefore stochastically disable the persistent-conditioning pathway during training... ... ", but they do not explain how exactly "disable" is implemented (set to zero? removed?). The reason why the CFG interval [0.1,1] differs from the PG interval [0,1] is also not explained. In addition, key hyperparameters such as the P-to-A dropout probability of 0.1 and the REPA weight of 0.1 are not ablated.
-
Some of the hyperparameter choices are also not transparent. The CFG scales are 2.8/2.7/3.0/2.9 and the PG scales are 3.0/2.8/1.9/2.0, but there is no explanation of how these scales were chosen.
-
The current manuscript lacks a discussion of limitations.
Questions for authors
-
Can the authors clarify whether heterogeneous refinement itself is necessary for the gains of PerF?
-
How is Persistence Guidance methodologically different from Register Guidance in "Registers Matter for Pixel Space Diffusion Transformers"?
-
Why are the main results reported with REPA rather than using PerF alone as the primary setting?
-
Could the authors report the actual sampling cost of PerF when PG is enabled?
-
Could the authors better justify the connection between compact representation and refinement budget?
-
Could the authors provide more details and ablations for the PG training and sampling configuration?
-
Could the authors discuss the main limitations of PerF?