- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Physis-Lang, a self-evolving framework that leverages physical language as a shared representation for data curation, model training, and video generation. The framework aims to improve the physical understanding of video world models through language-guided data retrieval and physical-aware learning.
Critical strengths
1、The proposed Physis-Lang framework provides a unified perspective by introducing physical language as a shared representation across data curation, model training, and video generation, which is a meaningful attempt to bridge physical reasoning and generative modeling.
2、The proposed language-guided data curation strategy, physical language representation, and self-evolving training paradigm are evaluated through multiple experiments, demonstrating the effectiveness of the framework in enhancing physical consistency and video generation capabilities.
Critical weaknesses
1、It is unclear whether the 71K videos selected from WISA-80K are obtained through the proposed language-guided retrieval strategy or through other filtering procedures. The authors should clarify the relationship between the proposed retrieval mechanism and the final training data construction pipeline.
2、To better demonstrate the effectiveness of the proposed language-guided retrieval mechanism, the authors could conduct additional experiments on a public physics-related video dataset. By retrieving different amounts of data using the proposed strategy and evaluating their impact on model training, the authors could more convincingly verify that the proposed method can select informative physical samples rather than relying on manually curated data.
3、The source and composition of the retrieval corpus used in the deficiency-guided retrieval pipeline are unclear. Although the authors state that 112K videos are collected through the proposed retrieval strategy, it is not specified from which video database these samples are retrieved, how large the candidate pool is, and how the retrieval corpus is constructed. More details are needed to better understand and reproduce the data construction process.
4、The proposed self-evolving retrieval pipeline relies on accurately identifying model deficiencies. However, the paper does not provide sufficient analysis of the deficiency diagnosis process. Since the identified deficiencies are directly used to generate retrieval queries, inaccurate diagnosis may lead to suboptimal data augmentation. The authors are encouraged to evaluate the reliability of deficiency identification and analyze how detection errors affect the retrieval results and model performance.
Questions for authors
see Weaknesses
5 comments
Author Response to Reviewer aPGG (2/2)
W3: Which video database these samples are retrieved, how large the candidate pool is, and how the retrieval corpus is constructed.
The candidate pool consists of 4M fetched YouTube video clips. They cover diverse categories, including conversations (65.4%), travel and outdoor scenes (11.8%), lifestyle and daily activities (10.7%), entertainment and media (5.5%), animals (3.4%), and educational and instructional content (3.3%).
The retrieval training dataset contains 111,792 unique videos across 14 physical-deficiency categories. We construct the dataset by processing categories in ascending order of their available candidate pool sizes, starting from the least represented category (low gravity), and collecting up to 10K unique videos per category. Its composition is as follows: human body motion, light, contact collision, rigid body motion, fluid dynamics, cloth deformation, physical material properties, and buoyancy each contribute 10K videos (8.95% per category, 71.56% combined). The remaining categories include chemical processes (9.4K, 8.41%), thermal processes (7.7K, 6.90%), fracture mechanics (6.2K, 5.56%), soft-body deformation (5.8K, 5.17%), elasticity (2.7K, 2.37%), and low gravity (26 videos, 0.02%). More details of how we constructed the 112K dataset are in Section 3.2.
W4: The authors are encouraged to evaluate the reliability of deficiency identification.
In our pipeline, the diagnosis results were manually reviewed before being used to construct retrieval queries. This human verification step ensured that the reported deficiency in each category was genuinely present in the corresponding video and prevented clearly inaccurate diagnoses from propagating into the retrieval and data-augmentation stages.
To demonstrate examples for the verification process, we have added Appendix I to the revised manuscript. The appendix presents a representative example for each deficiency category using six temporally ordered key frames, together with a brief explanation of the observed physical inconsistency. These examples provide direct evidence for the validity of the diagnosed deficiencies and clarify how each category is identified.
Paper revision
We have revised the manuscript, with all changes highlighted in blue. Specifically, we clarified the training data construction and filtering pipeline in Appendix B.5.1, and expanded Appendix I with examples of manually verified physical deficiencies.
Dear Reviewer aPGG,
With about one day left in the discussion period, we wanted to follow up on our rebuttal. Following your suggestions, we added retrieval experiments on the independent public PE-Video dataset at two data scales (20K and 117K), with improvements on both evaluated benchmarks. We also clarified the WISA filtering procedure, the retrieval corpus and its composition, and the manual verification of diagnosed physical deficiencies.
We would appreciate hearing whether these experiments and clarifications address your concerns. If any questions remain, we would be happy to discuss them further before the deadline.
Thank you again for your constructive feedback.
Best regards,
The Authors
Author Response to Reviewer aPGG (1/2)
Language-guided retrieval mechanism and general-purpose data filtering are two independent procedures in our training data construction pipeline. The 71K videos selected from WISA-80K are obtained through general quality filtering, not through the proposed language-guided retrieval mechanism.
Specifically, before training, we apply a general-purpose filtering stage to process training data. We compute motion scores using UniMatch optical flow [1] and aesthetic scores using the LAION aesthetic predictor [2], retaining only clips satisfying both criteria: (1 ≤ motion_score ≤ 70) and (aesthetic_score ≥ 0.7) (normalized to [0, 1]). We further use PySceneDetect [3] to identify and remove multi-shot videos, and exclude samples with incomplete metadata. This process yields 71K high-quality WISA videos from original WISA-80K. This filtering procedure was briefly described in Appendix B.5.1 of the original manuscript, and we have expanded this section in the revised manuscript to provide more detailed implementation details.
The two components play complementary roles: language-guided retrieval identifies videos targeting physical deficiencies in the pre-trained generator, while general-purpose filtering ensures the quality of training videos from both sources.
To further validate the effectiveness and generalizability of our proposed language-guided retrieval mechanism, we conduct additional experiments using PE-Video[4][5], which is a public video dataset consisting of 998k samples, independent of the original retrieval gallery. We re-run our language-guided retrieval pipeline on this new gallery to identify physics-informative training videos.
Specifically, we apply our retrieval pipeline to 14 physical-deficiency categories (we added Appendix I in the revised paper to visualize them), retrieving at most 10K videos per category. This yields a total of 117K newly retrieved videos. To further investigate the effect of the amount of retrieved training data, we uniformly sample 20K videos from this collection to construct a smaller training subset.
We keep the WISA-only training set (71K videos) fixed and vary the amount of newly retrieved data, comparing three settings: (1) WISA-only, without additional retrieved data; (2) WISA-only + Retrieved (20K), incorporating 20K newly retrieved videos; and (3) WISA-only + Full Retrieved (117K), incorporating the full set of 117K newly retrieved videos.
The results are summarized below:
The results demonstrate two key findings:
Effectiveness on a new public gallery. Incorporating only 20K newly retrieved videos improves performance over the WISA-only baseline by +1.67 points on PhyGenBench and +0.88 points on Physics-IQ Verified. Importantly, these videos are retrieved automatically from a new gallery, providing additional evidence that our method can identify useful physics-related training samples.
Consistent improvements with increasing retrieval scale. Increasing the retrieved data from 20K to 117K further improves performance, yielding overall gains of +3.55 points on PhyGenBench and +1.43 points on Physics-IQ Verified compared with the WISA-only baseline. The consistent improvements across both benchmarks demonstrate the benefits of incorporating additional physics-informative samples identified through our retrieval mechanism.
Overall, these experiments on a new, independently constructed public-data gallery provide further evidence of the effectiveness and generalizability of our language-guided retrieval mechanism, demonstrating that it can identify useful physics-related training data beyond the original retrieval gallery.
[1] H. Xu et al. Unifying Flow, Stereo and Depth Estimation, IEEE TPAMI, 2023.
[2] C. Schuhmann et al. LAION Aesthetic Predictor, 2022.
[3] B. Castellano et al. PySceneDetect, 2014.
[4] Daniel Bolya et al. Perception Encoder: The best visual embeddings are not at the output of the network, arXiv, 2025
[5] Jang Hyun Cho et al. PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding, arXiv, 2025