- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents PADiT for 360° video frame interpolation. Built on a latent diffusion Transformer, PADiT incorporates spherical positional encoding, ERP-aware pyramid feature fusion, RAFT-Lite-flow-guided confidence-weighted attention, and a motion-conditioned temporal-attention DiT. The model is trained by tokenizer pretraining followed by staged joint fine-tuning. The paper compares PADiT with several planar VFI methods on WEB360, 360VDS, and FlowScape, reporting improved LPIPS/FloLPIPS together with component ablations.
Critical strengths
1.The paper clearly identifies the latitude-dependent distortion and longitude-boundary continuity challenges induced by ERP projection. 2.The overall combination of geometry, cross-frame motion, and diffusion Transformer modeling is well motivated. 3.The component ablations indicate that the proposed modules contribute under the reported settings.
Critical weaknesses
1.The most relevant baseline is missing. The paper cites DM-360VFI, which is specifically designed for 360° diffusion-based VFI, but Table 1 only includes generic VFI methods that process ERP frames as planar inputs. Since PADiT's main claim concerns adaptation to 360° geometry, the absence of this direct baseline makes it difficult to assess the claimed novelty and the performance conclusion.
2.The main evaluation uses LPIPS, FloLPIPS, and limited PSNR computed on the ERP plane. Pixel sampling in ERP is non-uniform at high latitudes, so these planar metrics may not reflect spherical perceptual quality and do not directly test longitude seams or polar regions, which motivate the method. The paper does not include spherical/area-weighted or viewport-based metrics, nor region-wise results for seams and high latitudes. Consequently, the evidence does not yet establish that the geometric design resolves the claimed 360°-specific issues.
3.The method combines several existing ideas, including spherical position encoding, multi-scale fusion, flow warping/attention, motion conditioning, and staged training. However, the paper lacks a compute- and capacity-matched DiT baseline, and it does not report parameter counts, training cost, or the complete inference configuration. In particular, the submission omits the number of training iterations/epochs, batch size, hardware, whether RAFT-Lite is frozen or fine-tuned, sampling steps for each method, and the tuning protocol. This also makes it hard to assess the fairness of the retraining claim and the runtime/quality comparison.
4.Some cross-dataset gains are small (e.g., 360VDS FloLPIPS from 0.0239 to 0.0236 and LPIPS from 0.0129 to 0.0125), but the paper provides no uncertainty estimates or significance analysis. Since multiple triplets can come from the same video, at minimum the paper should report paired bootstrap confidence intervals using videos as the resampling unit, without treating correlated triplets as independent samples.
Questions for authors
1.Please add a direct comparison with DM-360VFI. If this is not feasible, please explain why and clarify the differences in data, input projection, training protocol, and model capacity. Are there other 360°-specific VFI methods that should also be included?
2.Can the authors report sphere-aware evaluation, such as spherical/area-weighted PSNR/SSIM, viewport-based LPIPS, or errors stratified by latitude band and proximity to the longitude seam? Do these results support the claim that SPE and flow-aware attention improve polar and boundary regions?
3.Please provide complete reproducibility and comparison details: training iterations or epochs, batch size, GPUs and total training cost, parameter counts, the training or freezing of RAFT-Lite, diffusion sampler and sampling steps, and the tuning budget for each retrained baseline. Please also include a capacity- and compute-matched DiT baseline to separate the benefit of the geometry modules from that of additional capacity or computation.
2 comments
Response to rJue Question 2 and 3
Response to Q2——Sphere-Aware Evaluation:
Thank you for this suggestion. PADiT is designed to improve the perceptual quality of interpolated frames, so our evaluation emphasizes LPIPS and FloLPIPS. Sphere-weighted PSNR/SSIM are useful measures of spherical pixel-level fidelity, but they do not directly measure perceptual quality. Moreover, the 360°-specific and diffusion-based methods in our comparison do not provide these metrics under a common evaluation protocol. Adding sphere-weighted scores for PADiT alone would therefore not enable a fair comparison with these methods.
To address the reviewer’s concern about regional behavior, we report PSNR stratified by longitude-seam proximity and latitude band. The seam region is defined as the leftmost and rightmost 5% of the ERP width. The latitude bands are defined as |lat| < 30°, 30° ≤ |lat| < 60°, and |lat| ≥ 60°.
| Region | WEB360 Dataset | 360VDS Dataset |
|---|---|---|
| PSNR (dB) | PSNR (dB) | |
| Full image | 37.65 | 35.21 |
| Seam region | 37.47 | 35.34 |
| Non-seam region | 37.72 | 35.19 |
| Equatorial band | 37.76 | 35.59 |
| Mid-latitude band | 37.58 | 35.27 |
| Polar band | 37.63 | 34.81 |
These regional PSNR results diagnose PADiT’s errors around the seam and across latitude bands. LPIPS and FloLPIPS remain our main perceptual metrics. Importantly, these regional measurements describe the full model’s behavior and cannot individually isolate the causal effect of SPE or flow-aware attention.
Response to Q3:
We appreciate these requests for reproducibility details and fair comparison. Below we enumerate all required configurations. All these specifications will be added to Appendix D, and we will release our full code for reproduction.
Q3_1. Training hyperparameters and compute resources
We train our PADiT framework on 4 NVIDIA RTX 3090 GPUs. Tokenizer pretraining ran for 300 epochs with a batch size of 16. The frozen-tokenizer DiT stage ran for 400 epochs with a batch size of 32, followed by 160 epochs of joint fine-tuning. Total training time is approximately 8 days.
Q3_2. Model parameter count
The MT-DiT uses a latent dimension of 16, hidden dimension of 768, 12 attention heads, 12 layers, and an MLP ratio of 4. The tokenizer uses a patch size of 16, hidden dimension of 768, 12 attention heads, and a latent dimension of 16, with eight encoder layers and eight decoder layers.
Q3_3. RAFT-Lite training configuration
RAFT-Lite is frozen throughout all training stages of PADiT. We only use it to extract flow features for the flow-aware attention module, without any parameter updates.
Q3_4. Diffusion sampler and sampling steps The PADiT uses velocity prediction with a two-step Euler ODE sampler and solver-time grid . The diffusion-baseline sampling budgets are two Euler ODE steps for EDEN, 50 steps for CBBD, and 200 DDIM steps for LDMVFI.
Q3_5. Tuning budget for retrained baselines
The retrained baselines follows their official settings. We did not conduct systematic hyperparameter searches for each baseline.
Q3_6. Capacity- and compute-matched DiT baseline
The existing DiT-based baselines compared in our main experiments are already close to our method in terms of model capacity and computational cost. Furthermore, we include a vanilla DiT baseline with matched parameter count in our ablation study.
We thank the reviewer for their careful and constructive comments again!
Response to rJue Question 1: 360° VFI Comparisons
Thank you for suggesting a comparison with DM-360VFI and for asking whether other 360°-specific VFI methods should be considered. We have added DM-360VFI to the quantitative comparison table, reporting its results alongside ours using the metrics shown in the table.
We also recognize SVI360 as a relevant 360°-specific VFI method and have added it to the related-work discussion. SVI360 evaluates middle-frame interpolation on FlowScape and reports PSNR, SSIM, spherical-weighted metrics (WS-PSNR and WS-SSIM), and optical-flow metrics. PADiT places particular emphasis on perceptual metrics, including LPIPS and FloLPIPS. Since SVI360 does not report LPIPS or FloLPIPS, its published results do not directly establish how it compares with PADiT on these perceptual measures.
Although SVI360 provides public code and pretrained weights, its released FlowScape checkpoint is trained on FlowScape, whereas PADiT is trained on WEB360. To compare the methods under a matched training-data protocol, SVI360 would need to be retrained on the WEB360 training split and then evaluated using the same test samples, input resolution, preprocessing, and metrics. We could not complete this retraining and evaluation within the revision period, and plan to investigate this comparison in future work.