- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Prior Subspace Recomposition (PSR), a framework for improving conditional generation through noise initialization. The method first discovers a set of high-quality noise anchors using downstream evaluation metrics. It then trains a lightweight ranker to predict the compatibility between these anchors and semantic conditions. During inference, the predicted preferences are aggregated into a global ranking, and the top anchors are combined to obtain the final initialization.
The method is evaluated on text-to-image, text-to-video, and image-to-video generation using several pretrained backbones. The experiments show consistent improvements over random initialization without modifying the underlying generative models.
Overall, I lean toward acceptance. The paper is clearly presented, the methodology is well explained, and the experiments provide reasonable support for the proposed approach. While I have some concerns about the computational overhead, I do not see any major technical issues.
I should also note that I am not an expert in this particular research area, and my knowledge of the related literature is limited. I am therefore not fully confident in judging the novelty of the contribution relative to prior work. My recommendation is mainly based on the clarity of the presentation and the experimental evidence, and I assign a relatively low confidence score.
Critical strengths
-
The paper is clearly written and well organized. Although I am not an expert in this specific research direction, I found the motivation and methodology easy to follow. The overall presentation is quite complete, and the technical details are sufficiently explained.
-
The experimental evaluation is fairly comprehensive, covering different generation tasks, backbones, and benchmarks. The reported improvements are consistent, and the ablation studies help explain the contributions of different components. Overall, I find the experimental results encouraging.
Critical weaknesses
My main concern is the additional computational cost. The method requires several offline stages, including anchor discovery and ranker training, and also introduces extra inference overhead. Although the authors provide an efficiency analysis, the practical trade-off between generation quality and computational cost could be discussed further.
Apart from this, I did not identify any major technical issues. However, my familiarity with the literature on noise initialization is limited, so I am less confident in assessing the contribution relative to existing work.
Questions for authors
-
How does the additional inference overhead scale with the number of anchors and the model size? Is there a recommended configuration for practical applications?
-
How sensitive is the method to the evaluation metrics used during anchor discovery? Would the selected anchors remain effective when evaluated using different metrics?
4 comments
Response to Question 2
For Question 2: Regarding Sensitivity to Evaluation Metrics and Cross-Metric Transfer
We thank the reviewer for this question. PSR is metric-adaptive rather than metric-invariant: the evaluation suite defines the objective for anchor discovery and ranking supervision, so changing that objective can affect which anchors are preferred. However, this does not imply that the resulting benefits are restricted to the metrics used during offline construction. The manuscript already investigates this question through two complementary experiments.
Transfer to non-optimized video-quality dimensions. In Appendix A.2, Table 5, we restrict both anchor discovery and semantic-conditioned supervision on Wan2.2-5B to TCS and MSS only, while evaluating generation using all five VMBench metrics. The common prompt is also revised accordingly to emphasize temporal coherence and motion smoothness.
| Configuration | PAS | OIS | TCS | MSS | CAS | Avg. |
|---|---|---|---|---|---|---|
| Random initialization | 14.86 | 68.65 | 95.21 | 79.46 | 58.58 | 63.35 |
| PSR, TCS+MSS only | 21.30 | 69.87 | 95.72 | 83.11 | 63.55 | 66.71 |
| PSR, full metric suite | 24.84 | 70.43 | 96.31 | 84.00 | 63.32 | 67.78 |
Although PAS, OIS, and CAS are excluded from anchor scoring and ranking supervision, they improve from 14.86 to 21.30, 68.65 to 69.87, and 58.58 to 63.55, respectively. The overall average increases from 63.35 to 66.71. This provides direct evidence that the benefits of the selected anchors and the resulting PSR pipeline extend to quality dimensions not used as offline supervision signals.
Using the full metric suite further improves the average score to 67.78, although the two-metric configuration retains a slightly higher CAS. Thus, the choice of objective affects the balance of improvements across dimensions, but the benefits are not confined to the explicitly optimized metrics.
Transfer to additional image-quality evaluators. The SDXL experiment in Table 4 (left) and Appendix A.4, Table 10 provides complementary evidence of cross-metric transfer. Both anchor discovery and semantic-conditioned ranking supervision use HPSv2 exclusively, while performance is evaluated using HPSv2, ImageReward, and CLIPScore. The results are reproduced below for convenience.
| Method | HPSv2 | ImageReward | CLIPScore |
|---|---|---|---|
| Random initialization | 29.71 | 96.63 | 0.8734 |
| NPNet (Golden Noise) | 29.88 | 98.81 | 0.8813 |
| PSR | 33.23 | 108.53 | 0.8991 |
PSR improves not only HPSv2 but also the two additional evaluation metrics: ImageReward increases from 96.63 to 108.53, and CLIPScore increases from 0.8734 to 0.8991, relative to random initialization. PSR also exceeds the reported NPNet scores on both metrics. These results indicate that the benefits of HPSv2-guided construction extend beyond the evaluator used to construct the anchor pool and supervise the ranker.
Together with the TCS+MSS-only video experiment, these results provide evidence of cross-metric transfer in both image and video generation. Our claim is not that one anchor pool is universally optimal for every evaluator. Rather, metric-guided construction can yield benefits beyond its offline objectives, while the same PSR procedure can be re-instantiated when a different evaluation objective is desired.
Follow-up Regarding Our Rebuttal for Submission #23595
Dear Reviewer Ly5E,
We hope this message finds you well. Thank you again for the time and thoughtful feedback you have devoted to our work.
We have posted a detailed response addressing each of your comments. Specifically, we clarified offline construction and inference costs, contextualized the quality–latency trade-off relative to a larger backbone, explained how overhead depends on anchor count and backbone inference cost, recommended a latency-sensitive configuration, and highlighted evidence of metric adaptability and cross-metric transfer in both image and video generation.
We would greatly appreciate hearing whether these clarifications and additional results have addressed your concerns. Should any questions remain, we would be pleased to provide further information during the author–reviewer discussion period.
Best regards,
The Authors of Submission #23595
Dear Reviewer,
The discussion period is nearing its end, and we haven’t yet seen a reply from you. We encourage you to respond to the authors’ comments, especially if you have further questions about points that may affect your assessment.
Thank you for taking the time to engage in the discussion.
Best,
Your AC
Response to Weakness 1 and Question 1
For Weakness 1 and Question 1: Regarding Computational Cost, Scaling, and Practical Configuration
We thank the reviewer for emphasizing the practical trade-off between generation quality and computational cost. This trade-off is examined in Section 4.3 and Table 4 (right) of the main manuscript, with additional details in Appendix A.3, Tables 8–9. We clarify below how the additional computation translates into generation-quality improvements and how the anchor budget can be adjusted for practical use.
Offline construction and reuse. On Wan2.1-1.3B using eight NVIDIA H200 GPUs, prior anchor discovery, concept–anchor supervision construction, and ranker training take approximately 3, 46, and 13 hours, respectively. These costs are incurred once for a given backbone and evaluation configuration. The resulting anchor pool and ranker are then reused across prompts, so repeated generation does not require repeating the offline pipeline. The construction cost can therefore be amortized over subsequent use within the same configuration.
Practical benefit of the additional computation. The additional inference cost allows the smaller backbone to approach the average generation quality of a substantially larger model. As reported in Appendix A.3, Wan2.1-1.3B with PSR achieves a VMBench average of 62.55, comparable to 63.35 for Wan2.2-5B with random initialization, while requiring approximately 115 seconds rather than 150 seconds per video. Under the reported model-specific generation configurations, this corresponds to a 0.80-point score difference and approximately 23.3% lower inference latency. Thus, the practical benefit is not merely an improvement over the same model’s random initialization: PSR also brings the smaller backbone closer to the larger model’s aggregate benchmark performance.
The existing results further demonstrate that the initialization budget is adjustable:
Reducing the anchor pool from 50 to 25 decreases the additional initialization latency by 45.8%, while the average score decreases by only 0.04 points. The 25-anchor configuration retains approximately 97.7% of the average-score improvement achieved with 50 anchors, providing a lower-latency operating point with nearly unchanged aggregate quality.
Scaling with anchor count and backbone inference cost. PSR’s main inference overhead comes from extracting prompt-conditioned first-step velocity features. The unconditional features associated with fixed anchors are precomputed and cached, while the prompt-conditioned features are evaluated in batches across anchors. The overhead is therefore more naturally understood relative to the backbone’s own inference time, rather than as a fixed delay or a function of parameter count alone. Both ordinary generation and PSR feature extraction use forward evaluations of the same backbone. Increasing the anchor count increases the feature-extraction workload, whereas reducing it lowers the additional cost. The exact latency ratio also depends on batching efficiency and the available hardware.
Recommended practical configuration. For practical deployment, we recommend setting the anchor budget relative to the full sampling-step count, rather than using a fixed number of anchors for every generator. An initial heuristic is to use approximately half as many anchors as sampling steps, together with Top-2 recomposition under the cached and batched implementation. Table 4 (right) illustrates the practical value of this reduced-budget configuration: with K=25, PSR adds 13 seconds to the 91-second generation baseline, corresponding to 14.3% overhead, while achieving 62.51 compared with 62.55 at K=50. Thus, K=25 is the recommended latency-sensitive configuration for the evaluated setting.