acceptodds
Desk rejected at ICLR 2027

CoFiTok: Coarse-to-Fine Denoising Tokens for Pixel-Space Diffusion

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion networks predict an image-sized noise field at each evaluation. We in- troduce CoFiTok, which organizes this field as an ordered sequence of continuous token contributions. Each token is synthesized independently through a shallow bias-free map, giving a zero token an exact zero-component meaning. Accumu- lating the components yields partial clean-image estimates at a fixed noisy state and diffusion time. Prefix and component losses train these estimates along a pre- scribed coarse-to-fine denoising path. We relate pixel-space prefix errors to noise- space errors and use fixed-component permutations to evaluate ordering while preserving the full prediction. Across eight dataset settings, CoFiTok consistently improves path alignment. Two-seed studies with 20,000 training steps reduce path AUC from 1.0098 to 0.0378 on Tiny ImageNet and from 0.9984 to 0.0379 on ImageNet-64 HF, with endpoint-MSE increases of 3.4% and 5.0%. On ImageNet- 256, the learned four-component order ranks first among all 24 permutations for both seeds. Ablations identify complementary effects of prefix and component su- pervision, alongside mixed endpoint and generation results. CoFiTok provides a structured interface for studying partial denoising within a diffusion prediction.

Primary area:
generative models
Keywords:
Diffusion Models, Denoising Tokens, Pixel-Space Diffusion, Prefix-Controllable Generation, Representation Factorization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vgSHOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

CoFiTok decomposes a pixel-space diffusion noise prediction into an ordered sequence of additive components synthesized from continuous tokens. Bias-free synthesis ensures that a zero token produces an exact zero component. Prefix and component losses supervise cumulative predictions along a prescribed coarse-to-fine path. Experiments report large reductions in path AUC, small endpoint degradation, and mixed generation-quality results.

Critical strengths
  1. The factorization is mathematically coherent. Bias-free synthesis guarantees Sk(0)=0S_k(0)=0, and the pixel-space/noise-space error relation is correct. Reordering already computed components preserves the final endpoint exactly, enabling a clean test of accumulation order.
  2. The controlled experiments are carefully constructed. Endpoint-only models use the same architecture, the prefix and component losses are ablated separately, and the trained four-component order ranks first among all 24 permutations for both tested seeds.
Critical weaknesses
  1. The central result primarily demonstrates compliance with a prescribed training target rather than independently useful coarse-to-fine structure. CoFiTok directly optimizes errors against the path qmq_m and is then evaluated using path AUC against that same path. Component supervision explicitly assigns ordered target increments, so the permutation experiment confirms that the model learned the imposed order. No independent semantic, spectral, perceptual, or downstream criterion establishes that this path is naturally coarse-to-fine or useful.
  2. No practical advantage of using prefixes is demonstrated. Endpoint reconstruction slightly worsens in the repeated studies, generation quality usually favors dense or existing diffusion baselines, and no latency, FLOP, or quality-versus-prefix-length result is reported. The experiments use very small models and short training budgets. Appendix S7 also states that the target interpolation/filter, loss reductions, token dimensions, predictor architecture, optimizer, noise schedule, and preprocessing are absent, preventing reproduction.
Questions for authors
  1. What is the exact definition of the target constructor QQ, including filtering, interpolation, normalization, timestep dependence, and loss reductions?
  2. Can the authors validate the ordering using an independent criterion not used during training, such as frequency content, semantic structure, human judgments, or a downstream prefix-use task?
  3. Does using a shorter prefix produce a meaningful compute-quality trade-off? Please report actual FLOPs and latency relative to a parameter- and compute-matched dense or additive multi-head baseline.
  4. Do the results persist at realistic model and training scale with standard 50,000-sample generation evaluation?
5 comments
AuthorsOpenReview ↗

Response to Reviewer vgSH (1/2)

We thank Reviewer vgSH for the constructive questions. Below we specify the target constructor and reference configuration (§1), report an independent Fourier-based ordering criterion (§2), present prefix operating points against a matched control (§3), and report a 50K-sample ImageNet-256 evaluation (§4).

1. What are the exact target constructor, loss reductions and reference protocol? (W2, Q1)

The box-pool target constructor is defined by spatial filtering, a power-law progression and its noise-space increments. Clean RGB images are normalized to [-1,1], and xt=αtx0+σtϵx_t=\alpha_t x_0+\sigma_t\epsilon, with αt=αˉt\alpha_t=\sqrt{\bar\alpha_t} and σt=1−αˉt\sigma_t=\sqrt{1-\bar\alpha_t}. For m<K, set

B_m(x_0)=\operatorname{Resize}_{H,W}\bigl(\operatorname{AvgPool}_{s_m,s_m}(x_0)\bigr),$$ and set $B_K(x_0)=x_0$. Pooling uses non-overlapping $s_m\times s_m$ windows and retains partial windows at the image border; resizing is bilinear with half-pixel-centred sampling. The constructor is $$Q(x_0,x_t,t,m)=q_m=(1-p_m)\frac{x_t}{\alpha_t}+p_mB_m(x_0), \qquad p_m=(m/K)^{1.5}.$$ Its exact endpoints are $q_0=x_t/\alpha_t$ and $q_K=x_0$. Targets retain their unclipped values in normalized image coordinates. The spatial filters and progress function are fixed across timesteps; timestep dependence enters through $x_t/\alpha_t$ and the corresponding noise-space conversion: $$e_m^*=\frac{x_t-\alpha_tq_m}{\sigma_t} =p_m\frac{x_t-\alpha_tB_m(x_0)}{\sigma_t},\qquad c_m^*=e_m^*-e_{m-1}^*,\quad e_0^*=0.$$ With $c_m=S_m(z^{(m)})$ and $E_m=\sum_{k\leq m}c_k$, the objective is $$\mathcal L=\operatorname{MSE}(E_K,\epsilon) +\frac{0.15}{K}\sum_{m=1}^K\operatorname{MSE}(E_m,e_m^*) +\frac{0.30}{K}\sum_{m=1}^K\operatorname{MSE}(c_m,c_m^*).$$ Each MSE averages over batch, channel, height and width. Both path losses average all K indices, including m=K. The reference configuration for the 64×64, K = 8 experiments on Tiny ImageNet and ImageNet-64 HF is: | Item | Reference setting | | --- | --- | | Tokens | Eight 16×64×64 token maps with 4, 4, 8, 8, 12, 12, 16 and 16 active channels | | Predictor | Small convolutional network; width 48; three trunk blocks; with token feedback | | Synthesis | Token-only, bias-free 1×1 then 3×3 convolutions; learned scalar per component | | Optimizer | AdamW; learning rate 2×10⁻⁴; weight decay 10⁻⁴; β = (0.9, 0.999); gradient clipping at 1 | | Training | Batch size 32; 20K steps; seeds 103/139 | | Diffusion | 1,000 steps; linear β schedule from 10⁻⁴ to 0.02 | | Preprocessing | RGB resize to 64×64; per-channel normalization with mean 0.5 and SD 0.5 | ### 2. Can an independent frequency criterion validate the ordering? (W1, Q2) An independent Fourier-ranking criterion gives **positive ordering gains in all 12 matched comparisons at t=50/250/500**. Its inputs are the predicted component spectra and component indices. Training uses endpoint, prefix and component MSE; the spectral ranking statistic is evaluated only on held-out predictions. For image i and component k, let $P_{i,k}(\omega)$ be channel-averaged power from an orthonormal Fourier transform. Its radial frequency centroid is $$\mu_{i,k}=\frac{\sum_{\omega}\|\omega\|_2 P_{i,k}(\omega)} {\sum_{\omega}P_{i,k}(\omega)},\qquad r_i=\operatorname{Spearman}\bigl((1,\ldots,K),(\mu_{i,1},\ldots,\mu_{i,K})\bigr).$$ Radial frequency is in cycles per pixel, and the denominator is floored at $10^{-20}$. The condition-level statistic is $\rho=N^{-1}\sum_i r_i$, directly quantifying the association between component position and frequency centroid. The matched 20K-step K = 8 models are compared on Tiny ImageNet and ImageNet-64 HF using seeds 103/139 and 1,024 class-stratified held-out images per condition, with paired images and noise. At t=500: | Dataset | Seed | CoFiTok ρ | Endpoint-only ρ | Paired gain [95% CI] | | --- | ---: | ---: | ---: | ---: | | Tiny ImageNet | 103 | 0.588 | 0.525 | +0.063 [0.059, 0.067] | | Tiny ImageNet | 139 | 0.793 | 0.500 | +0.293 [0.289, 0.296] | | ImageNet-64 HF | 103 | 0.685 | 0.621 | +0.064 [0.061, 0.068] | | ImageNet-64 HF | 139 | 0.869 | 0.524 | +0.345 [0.342, 0.347] | Gains are mean paired per-image correlation differences, CoFiTok minus endpoint-only. Each interval uses 1,000 paired image-bootstrap resamples conditional on the checkpoint pair. The same positive-gain result holds across all 12 conditions at t=50/250/500, with each paired 95% interval above zero. Across the full five-timestep grid t=50/250/500/750/950, CoFiTok's mean correlation is positive in all 20 dataset/seed/timestep conditions, ranging from 0.248 to 0.947. This provides a direct spectral characterization of the components, independent of the training target, together with paired evidence that path supervision strengthens frequency ordering.
AuthorsOpenReview ↗

Response to Reviewer vgSH (2/2)

3. What compute–quality operating points do prefixes provide against a matched control? (W2, Q3)

CoFiTok couples an executable prefix budget to a supervised intermediate-denoising trajectory. The reference is the same-architecture endpoint-only additive multi-head model: both models have 81,808 parameters, a full-prediction core cost of 0.6231G FLOPs, and a 20K-step training budget, which fixes model capacity and full-budget arithmetic for the comparison.

The sampler uses ϵ^t(m)=∑k≤mSk(zt(k))\hat\epsilon_t^{(m)}=\sum_{k\leq m}S_k(z_t^{(k)}) at every step of DDIM-50. The early-exit computation evaluates the shared trunk and the heads, feedback and synthesis needed for m; its output is identical to the corresponding prefix of the full forward pass (maximum absolute difference zero). For the same CoFiTok checkpoint, m=1 reduces counted core arithmetic by 15.3% relative to m=8. The operating points are:

Method / budgetCore FLOPs/forward (G)Forward latency (ms)Tiny ImageNet FID ↓ImageNet-64 HF FID ↓
CoFiTok, m=10.52764.32408.32 ± 27.59401.43 ± 23.49
CoFiTok, m=20.54125.28411.95 ± 34.14417.36 ± 16.30
CoFiTok, m=40.56856.42361.22 ± 30.33359.31 ± 2.43
CoFiTok, m=60.59586.37363.43 ± 40.73383.78 ± 88.17
CoFiTok, m=80.62318.03271.45 ± 5.86273.09 ± 10.07
Endpoint-only multi-head, m=80.62318.01268.05 ± 30.68255.10 ± 5.74

All generation rows use DDIM-50, identical per-index initial noise and 1,024 samples per configuration; FID is mean ± sample SD over training seeds 103/139. Compute and timing are measured on the Tiny ImageNet seed-139 checkpoints. Latencies are median single-forward times in FP32 at batch size 1 on an RTX 4090, from five synchronized runs of ten forward passes after five warm-up passes. FLOP counts cover convolutions, linear layers, attention products and fixed projections, with a multiply–add counted as two FLOPs.

The endpoint-only control supervises only the final prediction, so it offers a single full-budget operating point. CoFiTok additionally supervises the intermediate trajectory, so each m in its curve executes a defined partial prediction with measured arithmetic, latency and generation quality. Within this curve, m=8 gives the lowest mean FID on both datasets and m=1 gives the lowest core cost.

4. Do the results persist at scale under 50,000-sample generation evaluation? (W2, Q4)

At approximately 62.8M parameters and 100K training steps, CoFiTok achieves FID 112.53 versus 120.00 for its dense control, a 6.23% reduction, in a 50K-sample ImageNet-256 evaluation. Both models train on the full dataset with training seed 2027. CoFiTok and dense have 62,834,083 and 62,824,707 parameters, with full-prediction core costs of 571.024G and 570.439G FLOPs per denoiser forward. This closely matches capacity and counted forward arithmetic.

The generation comparison fixes EMA weights, full prediction budgets, DDIM-100, CFG=1.5, and identical initial noise and class labels for each sample index. Each model generates 50,000 images, evaluated against the same 50,000 validation images using the Inception-v3 features of torch-fidelity 0.4.0:

ModelFID ↓IS ↑KID ↓Precision ↑Recall ↑
CoFiTok112.539.874 ± 0.1360.07521 ± 0.001890.08390.3800
Dense120.009.298 ± 0.1110.07730 ± 0.002290.09370.3661

IS reports SD over 10 image splits; KID reports SD over 50 subsets of 1,000 features. These describe evaluator split and subset variability within each fixed checkpoint.

CoFiTok combines lower FID and KID with higher IS and recall in this matched pair. The precision–recall pairs characterize each model's coverage–fidelity operating point; CoFiTok's lower FID accompanies higher measured feature-space coverage.

Dear Reviewer,

The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.

Best, AC

Reviewer_ovE3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CoFiTok, which writes the noise prediction of a pixel-space diffusion network at a fixed (xt,t)(x_t,t) as a sum of KK components. A bias-free synthesis map ensures a zero token gives a zero component. Partial sums are converted into partial clean-image estimates and supervised against a prescribed path from xt/αtx_t/\alpha_t to x0x_0, using a prefix loss and a component (increment) loss in addition to the standard noise loss. Experiments use very small unconditional models with parameter-matched dense controls on eight dataset settings. Path supervision sharply reduces path error relative to endpoint-only factorization at a modest endpoint MSE cost, and the training order ranks first among permutations. Generation results are mixed and depend on the evaluator.

Critical strengths
  1. A clean, inspectable interface. Exposing a single dense prediction as an ordered, additive set of components, with an exactly zero-preserving token-to-component map and an endpoint-preserving permutation test, is a tidy design. It could support studies of partial denoising or adaptive computation.

  2. Well-controlled and candid internal comparisons. The endpoint-only control uses the same architecture, the dense control is parameter-matched (within 1.35% / 0.55%), and the ablations separate the two path losses. Negative results and missing details are reported openly.

Critical weaknesses
  1. The target path is unspecified, and the headline evidence shows fitting to that path rather than independent support for coarse-to-fine denoising.

    • Only the endpoints of the path are defined. No frequency ordering or monotonicity condition is stated, so "coarse-to-fine" is never defined and the targets cannot be reconstructed.
    • Path AUC measures error against the same intermediate targets that the path losses supervise, while the endpoint-only control receives no signal about them. The roughly 96% reduction (1.0098 to 0.0378 on Tiny ImageNet, Table 2) shows the prescribed path is learned and generalizes, which is expected, not that it is a useful decomposition.
    • Likewise, the order result (Table 3) shows alignment with the supervised accumulation order, and the pp sweep is relative to the fixed power-1.5 target.
    • The KK sweep is confounded: changing KK changes both the prefix grid of path AUC and the endpoint weight.
    • The component targets telescope to the prefix targets (Eq. 7), so the two losses carry the same target information, although combining them does improve path error.
    • Deep biased synthesis gives lower path AUC (0.0463 vs. 0.0686) but worse endpoint MSE, and it changes depth, nonlinearity and bias jointly. The main metric does not support the bias-free design by itself, and no experiment shows what zero preservation buys in practice.
  2. Practical value is not demonstrated, and the scale limits conclusions about diffusion models.

    • The models are tiny (81,808 / 15,776 parameters) and briefly trained, with endpoint PSNR around 13.5–15.2 dB.
    • Generation results are evaluator-dependent. CoFiTok is worse than its dense control on 7 of 8 low-resolution distances (Table 4) but better on 6 of 8 Inception-feature distances (Table S12). The larger ImageNet-256 control (Table S13) splits the same way.
    • The broad comparisons use only 512–1,024 samples without uncertainty estimates, and the external baselines are not matched in parameters, batch size or network evaluations.
    • Most importantly, shortened prefixes are never used in a sampler. No runtime or FLOPs are measured.
    • The repeated, order and ablation evaluations use only t=500t=500, the broad study's timestep is not stated, and there is no cross-timestep analysis.
    • The demonstrated contribution is control of the prescribed path. It does not yet establish useful coarse-to-fine behavior or a practical advantage.
Questions for authors

see the weakness part.

6 comments
AuthorsOpenReview ↗

Response to Reviewer ovE3 (1/2)

We thank Reviewer ovE3 for the detailed technical questions. This part addresses W1 (ordering, choice of K, loss design, zero preservation); Part 2 addresses W2 (scale, prefix execution, noise levels).

1. Coarse-to-fine definition and an independent ordering measure

The target path is specified in our reply to Reviewer Dh82 (§1): Bm(x0)B_m(x_0) is sm×sms_m\times s_m average pooling with sm=min⁡(2K−m,H,W)s_m=\min(2^{K-m},H,W) followed by bilinear upsampling, qm=(1−pm)xt/αt+pmBm(x0)q_m=(1-p_m)x_t/\alpha_t+p_mB_m(x_0) with pm=(m/K)1.5p_m=(m/K)^{1.5}, and prefix and component targets are the corresponding noise-space values.

To characterize ordering independently of this target, we compute the per-image Spearman correlation between component index and the radial Fourier-energy centroid of each predicted component; this statistic is not used in training. Across Tiny ImageNet and ImageNet-64 HF, seeds 103/139 and t = 50/250/500/750/950, CoFiTok's mean correlation is positive in all 20 conditions (0.248–0.947). Against the same-architecture endpoint-only control it is higher in all 12 comparisons at t = 50/250/500, with paired gains of +0.063 to +0.345 at t = 500 (1,024 held-out images, paired noise, all bootstrap 95% CIs above zero).

2. K comparison under a fixed target, grid and endpoint weight

Yes. K = 4/8/16 share one Gaussian-blur target (σblur(u)=81−u−1\sigma_{\mathrm{blur}}(u)=8^{1-u}-1 px, progress u1.5u^{1.5}) and are trained and evaluated on the same 16 positions uj=j/16u_j=j/16, with native prefixes linearly interpolated. The endpoint coefficient (1), the prefix average (15 positions) and the component average (16 increments) are identical across K. Tiny ImageNet, 5K steps, seeds 103/139, 1,024 images at t = 500:

KPath AUC ↓Endpoint MSE ↓
40.08670.1507
80.08020.1482
160.06680.1534

Path AUC decreases with K in both seeds; K = 16 reduces it by 23.0% relative to K = 4.

3. Why both prefix and component losses

With rm=Em−em∗r_m=E_m-e_m^* and r0=0r_0=0, telescoping gives cm−cm∗=rm−rm−1c_m-c_m^*=r_m-r_{m-1}. Hence Lp=1K∑m∥rm∥2\mathcal L_p=\frac1K\sum_m\|r_m\|^2 penalizes the cumulative error, while Lc=1K∑m∥rm−rm−1∥2\mathcal L_c=\frac1K\sum_m\|r_m-r_{m-1}\|^2 penalizes its change between positions. For example, residuals (a, a) and (a, −a) have the same Lp\mathcal L_p but component penalties 0.5a20.5a^2 and 2.5a22.5a^2. ImageNet-64 HF, K = 8, 5K steps, seeds 103/139, t = 500:

ObjectivePath AUC ↓Endpoint MSE ↓
Endpoint + prefix + component0.0686 ± 0.00170.1300 ± 0.0027
Endpoint + component0.1269 ± 0.00960.1300 ± 0.0003
Endpoint + prefix0.1105 ± 0.00130.1302 ± 0.0012

The joint objective gives the lowest path AUC in both seeds (45.7% and 37.9% lower than the two single-term variants), with endpoint MSE within 0.2%.

4. Zero preservation, independent of depth and nonlinearity

For a binary mask, ∑kSk(mkz(k))−∑k:mk=1Sk(z(k))=∑k:mk=0Sk(0)\sum_k S_k(m_kz^{(k)})-\sum_{k:m_k=1}S_k(z^{(k)})=\sum_{k:m_k=0}S_k(0), which vanishes for bias-free synthesis; zero-masking a token is then exactly equivalent to removing its contribution. To isolate the bias, we compare the same shallow K = 8 synthesis with and without zero-initialized trainable biases, with all other settings matched (Tiny ImageNet, 5K steps, t = 500, prefixes m = 1/4/8):

SynthesisSeedMax. zero-token outputMax. masking/omission gap
Bias-free103 / 1390 / 00 / 0
With biases1030.029830.04223
With biases1390.014970.04301

With biases the gap is nonzero in both seeds, so component ablations would no longer correspond to exact removal.

AuthorsOpenReview ↗

Response to Reviewer ovE3 (2/2)

5. Larger models and standard sample counts

On full ImageNet-256, CoFiTok and a dense control (62.83M vs. 62.82M parameters; 571.0G vs. 570.4G FLOPs per forward) are trained for 100K steps (seed 2027) and sampled with EMA weights, DDIM-100, CFG = 1.5 and identical noise and class labels. Each model generates 50,000 images, evaluated against the 50,000 validation images (torch-fidelity 0.4.0):

ModelFID ↓IS ↑KID ↓Precision ↑Recall ↑
CoFiTok112.539.8740.075210.08390.3800
Dense120.009.2980.077300.09370.3661

CoFiTok attains 6.23% lower FID, lower KID, higher IS and higher recall at matched capacity and computation.

6. Shortened prefixes in sampling

Yes. Every DDIM-50 step uses ϵ^t(m)=∑k≤mSk(zt(k))\hat\epsilon_t^{(m)}=\sum_{k\le m}S_k(z_t^{(k)}) with m fixed per trajectory. Early exit evaluates the shared trunk once and then only the heads, feedback and synthesis of the first m components; its output is identical to the corresponding prefix of the full forward pass. In the 81,808-parameter model (two seeds, 1,024 samples each; latency in FP32 at batch size 1 on an RTX 4090):

mFLOPs (G)Latency (ms)Tiny ImageNet FID ↓ImageNet-64 HF FID ↓
10.52764.32408.32 ± 27.59401.43 ± 23.49
20.54125.28411.95 ± 34.14417.36 ± 16.30
40.56856.42361.22 ± 30.33359.31 ± 2.43
60.59586.37363.43 ± 40.73383.78 ± 88.17
80.62318.03271.45 ± 5.86273.09 ± 10.07

All budgets come from the same checkpoint; m = 1 reduces core FLOPs by 15.3% relative to m = 8.

7. Persistence across noise levels

We evaluate the matched 20K-step K = 8 CoFiTok and endpoint-only models on Tiny ImageNet and ImageNet-64 HF (seeds 103/139) at five timesteps, using the same 1,024 images and noise throughout:

tΔAUC 95% CI < 0Trained order ranked 1st of 18Frequency Spearman (range)
504/44/40.833–0.862
2504/44/40.910–0.947
5004/44/40.588–0.869
7504/44/40.259–0.541
9504/44/40.248–0.553

In all 20 conditions, CoFiTok has lower path AUC than the control (bootstrap 95% CI of ΔAUC below zero), and the trained order ranks first among 18 orders (trained, reverse and 16 fixed random orders).

Dear Reviewer,

The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.

Best, AC

Reviewer_Dh82OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CoFiTok factors each pixel-space diffusion noise prediction into ordered continuous token contributions. A token-only synthesis map makes zero tokens produce zero components. Prefix and component losses supervise partial image estimates along a prescribed denoising path. Experiments compare endpoint-only factorization and dense predictors across eight settings. Two-seed studies show large reductions in target-path AUC, with modest endpoint-error increases. Fixed-component permutations support the learned accumulation order. Generation results remain mixed. The demonstrated contribution is chiefly an interface for exposing and supervising partial predictions.

Critical strengths
  1. The paper clearly separates components within one network evaluation from successive sampler steps. This creates a useful interface for partial predictions. Zero-preserving synthesis gives token removal an exact meaning in noise space.

  2. The supervision comparison uses identical architectures. Tiny ImageNet path AUC falls from 1.0098 to 0.0378 after 20k steps. The paper also reports the accompanying 3.4% endpoint-error increase, making the tradeoff clear (Table 2).

  3. The permutation test changes accumulation order without changing the full prediction. The trained order ranks first among all 24 orders for both seeds. The loss ablations further support the value of supervising prefixes and components together (Tables 3 and 5).

Critical weaknesses
  1. Section 3.3 does not fully define the prescribed target path. Its filter, interpolation, and loss reductions remain unspecified. Both training and path AUC depend on this construction. The missing definition therefore limits interpretation and reproduction of the central result.

  2. The demonstrated benefit mainly concerns agreement with the training path. Endpoint errors often increase, and generation results are mixed. The paper acknowledges this tradeoff. However, the practical value of the exposed prefixes remains uncertain beyond inspecting the prescribed path.

Questions for authors
  1. Section 3.3 makes the target path central to both training and evaluation. Could you specify the intermediate-image construction, including its filter and interpolation? Please also provide the exact prefix and component loss reductions. These details would clarify the meaning of coarse-to-fine denoising and make the AUC results reproducible.

  2. Tables 2 and 4 show better path alignment but mixed endpoint and generation outcomes. What concrete use follows for exposed prefixes from the existing results? Please distinguish target-path fidelity from sampling quality and computation savings. This would clarify the present contribution without requiring additional experiments.

3 comments

We thank Reviewer Dh82 for the careful review and for recognizing the separation of within-evaluation components from sampler steps, the identical-architecture supervision comparison and the fixed-endpoint permutation test. Below we specify the target-path construction and loss reductions (§1), and describe the concrete operations that exposed prefixes provide, distinguishing path fidelity, sampling quality and computation (§2).

1. How is the target path constructed, and how are the losses reduced? (W1, Q1)

For normalized RGB images in [-1,1], write xt=αtx0+σtϵx_t=\alpha_t x_0+\sigma_t\epsilon, where αt=αˉt\alpha_t=\sqrt{\bar\alpha_t} and σt=1−αˉt\sigma_t=\sqrt{1-\bar\alpha_t}. For m<Km<K, define

B_m(x_0)=\operatorname{Bilinear}_{H,W} \bigl(\operatorname{AvgPool}_{s_m,s_m}(x_0)\bigr),$$ and set $B_K(x_0)=x_0$. Average pooling uses non-overlapping $s_m\times s_m$ windows and retains partial windows at the image border; bilinear resizing uses half-pixel-centred sampling. The prescribed path is $$q_0=x_t/\alpha_t,\qquad p_m=(m/K)^{1.5},\qquad q_m=(1-p_m)q_0+p_mB_m(x_0).$$ No clipping is applied in this target constructor. Its timestep dependence is through $q_0,\alpha_t,\sigma_t$; the spatial filters and progress schedule are independent of $t$. The prefix-noise and component targets are $$e_m^*=(x_t-\alpha_tq_m)/\sigma_t =p_m(x_t-\alpha_tB_m(x_0))/\sigma_t,\qquad c_m^*=e_m^*-e_{m-1}^*,\quad e_0^*=0.$$ Writing $E_m=\sum_{k\leq m}S_k(z^{(k)})$, the losses are $$\mathcal L=\operatorname{MSE}(E_K,\epsilon) +\frac{0.15}{K}\sum_{m=1}^K\operatorname{MSE}(E_m,e_m^*) +\frac{0.30}{K}\sum_{m=1}^K\operatorname{MSE}(S_m(z^{(m)}),c_m^*).$$ Each MSE averages over batch, channel, height and width. The path AUC reported in the paper is the trapezoidal average of prefix image-space MSE over $m/K\in[1/K,1]$. Thus it measures agreement with this specified box-pool/interpolation path. The construction does not impose semantic ordering or an ideal frequency cutoff; frequency ordering is measured separately by a spectral statistic (reply to Reviewer vgSH, §2). ### 2. What concrete uses do exposed prefixes provide? (W2, Q2) CoFiTok provides a denoising interface with three directly executable operations: selecting a partial prediction, removing a component with a known effect, and changing accumulation order at a fixed endpoint. These operations make the construction of a single noise prediction accessible to controlled analysis. Holding the computed tokens fixed, zeroing token $k$ removes exactly $S_k(z^{(k)})$, changing the clean-image estimate by $(\sigma_t/\alpha_t)S_k(z^{(k)})$. Permuting the same components preserves their full sum and isolates the effect of accumulation order on intermediate predictions. The endpoint-only objective supervises the final sum; CoFiTok additionally supervises the trajectory of partial predictions. The matched-architecture Tiny ImageNet comparison quantifies this trajectory control: path AUC falls from 1.0098 to 0.0378, a 96.3% reduction. In the ImageNet-256 K = 4 order study, the trained order ranks first among all 24 permutations for both seeds. The three evaluation axes have distinct roles: path AUC measures control of the intermediate trajectory, FID measures the generated distribution at a stated sampling budget, and FLOPs measure execution cost. For the ImageNet-256 model at full prefix budget, the 50K-sample evaluation gives FID 112.53, 6.23% lower than its dense control's 120.00 under the same DDIM-100/CFG-1.5 protocol. Prefix length also supplies an explicit execution budget. In the small K = 8 model, an exactly equivalent early-exit computation reduces counted core FLOPs from 0.6231G at m=8 to 0.5276G at m=1, a 15.3% reduction. Each prefix retains its defined partial-denoising interpretation, so the budget can be chosen per use without retraining.

Dear Reviewer,

The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.

Best, AC

AuthorsOpenReview ↗

Thank You for Your Consideration and Follow-Up

Dear Reviewer,

We sincerely thank you for your thoughtful review and constructive feedback. In our rebuttal, we have made every effort to address your concerns by clarifying the target construction and providing additional experimental evidence on component ordering, computational efficiency, and generation performance.

As the discussion period draws to a close, we would be very grateful if you could kindly review our responses and share your thoughts. Should any concerns remain, we would be more than happy to provide further clarification.

Thank you again for your consideration and support!

Best regards,

Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

A dense noise estimate exposes a final prediction while leaving its intermediate contributions implicit. CoFiTok makes a single denoiser evaluation accessible as an ordered additive interface: partial predictions can be supervised, and token-only zero-preserving synthesis gives component interventions an exact meaning. Three results support this design:

  • Generation at matched capacity. In the full-data ImageNet-256 comparison, the approximately 62.8M-parameter models train for 100K steps with seed 2027. CoFiTok achieves FID 112.53 versus 120.00 for dense, a 6.23% reduction, at closely matched parameter count and counted forward arithmetic. Each model generates 50,000 images under the same EMA/DDIM-100/CFG=1.5 protocol and evaluator. The individual replies report all five generation metrics and their evaluation settings.
  • Independent component ordering. On Tiny ImageNet and ImageNet-64 HF, seeds 103/139 give stronger Fourier-based frequency ordering than the matched endpoint-only control in all 12 dataset/seed/timestep comparisons at t=50/250/500; every paired image-bootstrap 95% interval for the ordering gain is above zero. Across t=50/250/500/750/950, CoFiTok's mean ordering correlation is positive in all 20 conditions. These measurements characterize predicted component spectra independently of target-path AUC.
  • Exact interventions and executable budgets. Holding the computed tokens fixed, zeroing token k removes exactly its contribution S_k(z^(k)). Prefixes m=1/2/4/6/8 are executable at every DDIM-50 step, and early exit reproduces the corresponding prefix of the full forward pass exactly. In the 81,808-parameter model, m=1 reduces counted core FLOPs by 15.3%, from 0.6231G to 0.5276G relative to m=8. The individual replies report the full quality–cost curves and measured forward latencies.

The individual replies specify the target constructor, loss reductions and reference configuration, together with common-target K controls and matched synthesis and loss ablations.

AuthorsOpenReview ↗

Sincere Appreciation and Overall Response to the Area Chair and Reviewers

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all reviewers for their thoughtful feedback and constructive discussions. Throughout the rebuttal, we have made every effort to address the concerns through additional experiments, methodological clarifications, and more rigorous evaluations. We also appreciate the reviewers' recognition of our progress and their suggestions for further improvement.

First, we have further clarified and validated the core contributions of CoFiTok. Beyond learning a prescribed denoising trajectory, our framework provides an ordered representation that enables exact component interventions. This property is supported by controlled ablations, while independent Fourier analysis confirms positive frequency-ordering gains across all 12 matched comparisons. Furthermore, additional experiments across noise levels and token configurations strengthen our findings, alongside more detailed methodological specifications to improve reproducibility.

Building on these findings, we have also provided stronger evidence of practical utility and scalability. Prefix-based inference achieves up to a 15.3% reduction in counted core FLOPs, although shorter prefixes involve generation-quality trade-offs. More importantly, our 50,000-sample ImageNet-256 evaluation with approximately 62.8M-parameter models achieves an FID of 112.53, compared with 120.00 for the matched dense baseline, representing a 6.23% improvement. These results further support the potential of CoFiTok beyond prescribed-path alignment.

Taken together, the additional evidence reinforces CoFiTok's contribution as a structured and intervention-capable denoising framework. Meanwhile, we recognize the remaining limitations and will carefully incorporate the new results, clarify our claims, and strengthen the discussion in the revised manuscript.

Finally, we sincerely appreciate the reviewers' and Area Chair's time, consideration, and valuable suggestions, which have significantly strengthened our work. We hope that our responses and additional evidence will help inform the final assessment, and we remain grateful for your consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.