acceptodds
Under review as a conference paper at ICLR 2027

Panorama-Aware Diffusion Transformer for 360° Video Frame Interpolation

Abstract

Panoramic Video Frame Interpolation (VFI) poses unique challenges due to the equirectangular projection (ERP), which leads to latitude-dependent spatial distortion and a discontinuous longitude boundary. Furthermore, the presence of large and nonlinear motion in ERP panoramas complicates correspondence estimation and temporally coherent synthesis. In this work, we present PADiT, the first Panorama-Aware Diffusion Transformer-based framework specifically designed for video frame interpolation. PADiT introduces new DiT blocks that blend Motion perception and Temporal-attention (MT-DiT), where each transformer block employs multi-head temporal attention to aggregate information from the input ERP frame pair and the latent representation of the intermediate frame. We also integrate flow-derived motion embeddings with diffusion timesteps and frame differences, allowing for the modulation of the denoising blocks through adaptive layer normalization. To enhance geometric and motion modeling for ERP panoramas, we implement spherical position encoding, flow-aware attention, and panorama-aware pyramid feature fusion. These components help mitigate issues associated with ERP, such as positional bias, unreliable motion correspondences, and the misalignment of cross-scale features.Experiments on several widely used panoramic benchmarks, including WEB360, 360VDS, and FlowScape, demonstrate that the proposed PADiT outperforms state-of-the-art methods in panoramic VFI. Specifically, it reduces the FloLPIPS and LPIPS metrics by up to 9.33% and 8.44%, respectively.

Primary area:
generative models
Keywords:
360-degree video, Diffusion transformer, Frame interpolation, Flow-aware attention

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_rJueOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents PADiT for 360° video frame interpolation. Built on a latent diffusion Transformer, PADiT incorporates spherical positional encoding, ERP-aware pyramid feature fusion, RAFT-Lite-flow-guided confidence-weighted attention, and a motion-conditioned temporal-attention DiT. The model is trained by tokenizer pretraining followed by staged joint fine-tuning. The paper compares PADiT with several planar VFI methods on WEB360, 360VDS, and FlowScape, reporting improved LPIPS/FloLPIPS together with component ablations.

Critical strengths

1.The paper clearly identifies the latitude-dependent distortion and longitude-boundary continuity challenges induced by ERP projection. 2.The overall combination of geometry, cross-frame motion, and diffusion Transformer modeling is well motivated. 3.The component ablations indicate that the proposed modules contribute under the reported settings.

Critical weaknesses

1.The most relevant baseline is missing. The paper cites DM-360VFI, which is specifically designed for 360° diffusion-based VFI, but Table 1 only includes generic VFI methods that process ERP frames as planar inputs. Since PADiT's main claim concerns adaptation to 360° geometry, the absence of this direct baseline makes it difficult to assess the claimed novelty and the performance conclusion.

2.The main evaluation uses LPIPS, FloLPIPS, and limited PSNR computed on the ERP plane. Pixel sampling in ERP is non-uniform at high latitudes, so these planar metrics may not reflect spherical perceptual quality and do not directly test longitude seams or polar regions, which motivate the method. The paper does not include spherical/area-weighted or viewport-based metrics, nor region-wise results for seams and high latitudes. Consequently, the evidence does not yet establish that the geometric design resolves the claimed 360°-specific issues.

3.The method combines several existing ideas, including spherical position encoding, multi-scale fusion, flow warping/attention, motion conditioning, and staged training. However, the paper lacks a compute- and capacity-matched DiT baseline, and it does not report parameter counts, training cost, or the complete inference configuration. In particular, the submission omits the number of training iterations/epochs, batch size, hardware, whether RAFT-Lite is frozen or fine-tuned, sampling steps for each method, and the tuning protocol. This also makes it hard to assess the fairness of the retraining claim and the runtime/quality comparison.

4.Some cross-dataset gains are small (e.g., 360VDS FloLPIPS from 0.0239 to 0.0236 and LPIPS from 0.0129 to 0.0125), but the paper provides no uncertainty estimates or significance analysis. Since multiple triplets can come from the same video, at minimum the paper should report paired bootstrap confidence intervals using videos as the resampling unit, without treating correlated triplets as independent samples.

Questions for authors

1.Please add a direct comparison with DM-360VFI. If this is not feasible, please explain why and clarify the differences in data, input projection, training protocol, and model capacity. Are there other 360°-specific VFI methods that should also be included?

2.Can the authors report sphere-aware evaluation, such as spherical/area-weighted PSNR/SSIM, viewport-based LPIPS, or errors stratified by latitude band and proximity to the longitude seam? Do these results support the claim that SPE and flow-aware attention improve polar and boundary regions?

3.Please provide complete reproducibility and comparison details: training iterations or epochs, batch size, GPUs and total training cost, parameter counts, the training or freezing of RAFT-Lite, diffusion sampler and sampling steps, and the tuning budget for each retrained baseline. Please also include a capacity- and compute-matched DiT baseline to separate the benefit of the geometry modules from that of additional capacity or computation.

2 comments
AuthorsOpenReview ↗

Response to rJue Question 1: 360° VFI Comparisons

Thank you for suggesting a comparison with DM-360VFI and for asking whether other 360°-specific VFI methods should be considered. We have added DM-360VFI to the quantitative comparison table, reporting its results alongside ours using the metrics shown in the table.

We also recognize SVI360 as a relevant 360°-specific VFI method and have added it to the related-work discussion. SVI360 evaluates middle-frame interpolation on FlowScape and reports PSNR, SSIM, spherical-weighted metrics (WS-PSNR and WS-SSIM), and optical-flow metrics. PADiT places particular emphasis on perceptual metrics, including LPIPS and FloLPIPS. Since SVI360 does not report LPIPS or FloLPIPS, its published results do not directly establish how it compares with PADiT on these perceptual measures.

Although SVI360 provides public code and pretrained weights, its released FlowScape checkpoint is trained on FlowScape, whereas PADiT is trained on WEB360. To compare the methods under a matched training-data protocol, SVI360 would need to be retrained on the WEB360 training split and then evaluated using the same test samples, input resolution, preprocessing, and metrics. We could not complete this retraining and evaluation within the revision period, and plan to investigate this comparison in future work.

MethodWEB360360VDSFlowScape
PSNR ↑FloLPIPS ↓LPIPS ↓RT (s) ↓FloLPIPS ↓LPIPS ↓RT (s) ↓FloLPIPS ↓LPIPS ↓RT (s) ↓
EDEN34.4060.04200.02231.8500.02560.01511.0460.04150.02681.617
CBBD37.0420.03080.01541.9100.02980.01481.2820.04050.01921.806
LDMVFI35.8370.04490.02145.6070.02640.01394.0050.04220.02396.832
MOMO36.8090.03260.01700.7740.02390.01290.4530.02680.01330.669
AMT37.3100.05330.02560.4330.02920.01650.3780.08240.05300.405
DM-360VFI38.0260.05780.02311.4020.04210.01960.6380.04900.02541.153
Ours37.6460.02800.01410.9780.02360.01250.4750.02430.01290.908
AuthorsOpenReview ↗

Response to rJue Question 2 and 3

Response to Q2——Sphere-Aware Evaluation:

Thank you for this suggestion. PADiT is designed to improve the perceptual quality of interpolated frames, so our evaluation emphasizes LPIPS and FloLPIPS. Sphere-weighted PSNR/SSIM are useful measures of spherical pixel-level fidelity, but they do not directly measure perceptual quality. Moreover, the 360°-specific and diffusion-based methods in our comparison do not provide these metrics under a common evaluation protocol. Adding sphere-weighted scores for PADiT alone would therefore not enable a fair comparison with these methods.

To address the reviewer’s concern about regional behavior, we report PSNR stratified by longitude-seam proximity and latitude band. The seam region is defined as the leftmost and rightmost 5% of the ERP width. The latitude bands are defined as |lat| < 30°, 30° ≤ |lat| < 60°, and |lat| ≥ 60°.

RegionWEB360 Dataset360VDS Dataset
PSNR (dB)PSNR (dB)
Full image37.6535.21
Seam region37.4735.34
Non-seam region37.7235.19
Equatorial band37.7635.59
Mid-latitude band37.5835.27
Polar band37.6334.81

These regional PSNR results diagnose PADiT’s errors around the seam and across latitude bands. LPIPS and FloLPIPS remain our main perceptual metrics. Importantly, these regional measurements describe the full model’s behavior and cannot individually isolate the causal effect of SPE or flow-aware attention.

Response to Q3:

We appreciate these requests for reproducibility details and fair comparison. Below we enumerate all required configurations. All these specifications will be added to Appendix D, and we will release our full code for reproduction.

Q3_1. Training hyperparameters and compute resources

We train our PADiT framework on 4 NVIDIA RTX 3090 GPUs. Tokenizer pretraining ran for 300 epochs with a batch size of 16. The frozen-tokenizer DiT stage ran for 400 epochs with a batch size of 32, followed by 160 epochs of joint fine-tuning. Total training time is approximately 8 days.

Q3_2. Model parameter count

The MT-DiT uses a latent dimension of 16, hidden dimension of 768, 12 attention heads, 12 layers, and an MLP ratio of 4. The tokenizer uses a patch size of 16, hidden dimension of 768, 12 attention heads, and a latent dimension of 16, with eight encoder layers and eight decoder layers.

Q3_3. RAFT-Lite training configuration

RAFT-Lite is frozen throughout all training stages of PADiT. We only use it to extract flow features for the flow-aware attention module, without any parameter updates.

Q3_4. Diffusion sampler and sampling steps The PADiT uses velocity prediction with a two-step Euler ODE sampler and solver-time grid 0,0.75,10, 0.75, 1. The diffusion-baseline sampling budgets are two Euler ODE steps for EDEN, 50 steps for CBBD, and 200 DDIM steps for LDMVFI.

Q3_5. Tuning budget for retrained baselines

The retrained baselines follows their official settings. We did not conduct systematic hyperparameter searches for each baseline.

Q3_6. Capacity- and compute-matched DiT baseline

The existing DiT-based baselines compared in our main experiments are already close to our method in terms of model capacity and computational cost. Furthermore, we include a vanilla DiT baseline with matched parameter count in our ablation study.

We thank the reviewer for their careful and constructive comments again!

Reviewer_7RvROpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper deals with diffusion-based Panoramic Video Frame Interpolation (VFT) problem. Previous works are mainly designed for interpolation of planar frames and thus their performance may degenerate in the presence of large and nonlinear motion in ERP panoramas. This paper present the first Panorama-Aware Diffusion Transformer-based framework specifically designed for 360◦ video frame interpolation. The authors incorporate the prior of panoramic coordinates into the designs of multiple components, e.g., the spherical position encoding, the Panoramic-Aware Pyramid Feature Fusion, etc. Besides, the authors propose a two-stage optimization strategy to progressively train the interpolation diffusion DiT. Experiment results verify the effectiveness of proposed method.

Critical strengths
  • The setting is meaningful and has not been explored previously.

  • The design considers the specific characteristic of Panoramic Video Frame Interpolation and is reasonable.

  • The empirical results are good and the ablations justify the effectiveness of each component.

Critical weaknesses

No critical weakness. Some questions to be addressed are listed in the following part.

Questions for authors
  • I didn't quite understand the motivation and the effect of flow-aware attention compared to the conventional attention. In my understanding, the authors firstly warp the video frames according to the optical flow and then perform the attention. As changing the order of tokens does not affect the attention results, why can flow-based warping be better? Please give more explanations.

  • Some typo exists. E.g., in Eq. (8), the subscript for the last loss term should be "rec" rather than "reg". Please check throughout the paper to fix all the typos.

1 comment
AuthorsOpenReview ↗

Response to 7RvR

We greatly appreciate the reviewer’s careful assessment and positive feedback on our work. Below we provide our point-by-point responses.

Response to Q1——Motivation and Effect of Flow-Aware Attention:

Thank you for raising this question. The reviewer is correct that simply permuting tokens, while consistently permuting their features and positional information, does not change the self-attention result.

Flow-based warping is not merely a token permutation. It resamples feature values at spatially displaced coordinates, typically using bilinear interpolation. This changes the feature content at each target location and may combine neighboring source features; in general, it is not a one-to-one permutation. We use bidirectional RAFT-Lite flow to project the two input feature maps toward the target time. This provides attention with features that are better spatially aligned with the estimated content of the intermediate frame, allowing attention to refine and fuse features around corresponding target locations. The motivation is therefore to provide a motion-based correspondence prior, rather than to exploit the order sensitivity of attention.The ablation study in the paper already evaluates flow-aware attention by comparing it with the corresponding model without this module.

Response to Q2——Correction of Eq. (8) and Manuscript Proofreading:

Thank you for identifying this error. We have corrected the subscript of the final loss term in Eq. (8) from LregL_{\mathrm{reg}} to LrecL_{\mathrm{rec}}, consistent with the reconstruction loss described in the text. We have also proofread the entire manuscript, including the equations, figure captions, tables, and supplementary material, and corrected other typographical errors and notation inconsistencies we identified.

Reviewer_y9wjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a special transformer architecture design and add optical flow as additional augmentation signal. Spherical encoding is introduced here to solve the problem for 360 videos.

Critical strengths
  1. The encoding strategy makes sense and seem to be practical in use
  2. The flow-aware attention is an interesting design that uses auxiliary signals to assist generation.
Critical weaknesses
  1. Qualitative comparisons are limited.
  2. Failure cases and limitation are not discussed.
  3. Some implementation details are not reported.
Questions for authors
  1. Could authors illustrate on why tokenizer pretraining is needed? I don't fully understand why it is needed.
  2. Could you provide GPU usage on training and inference
  3. Could you provide more qualitative cases, current cases are not persuasive
  4. Failure cases and discussion of limitation is expected I would raise my score if my questions are addressed
3 comments
AuthorsOpenReview ↗

Response to y9wj

Response to Q1——Motivation for Tokenizer Pretraining:

Thank you for raising this question. We appreciate the opportunity to clarify the role of tokenizer pretraining.

Tokenizer pretraining gives the diffusion transformer a learned, compact latent space before the interpolation model is optimized. The tokenizer is trained to encode and reconstruct panoramic frames, so the subsequent transformer predicts intermediate content in a representation adapted to the image statistics rather than directly modeling every pixel. This staged setup also separates representation learning from conditional interpolation optimization and allows the tokenizer to be frozen during the initial DiT training stage. We will explain this motivation and the training stages more clearly.

Response to Q2——GPU Usage and Runtime:

Thank you for asking us to clarify the GPU usage during training and inference.

Training was conducted on a server with four NVIDIA RTX 3090 GPUs. To the best of our recollection, tokenizer pretraining, DiT training with the tokenizer frozen, and joint fine-tuning each took more than one week. Exact wall-clock times, total GPU-hours, and peak GPU memory were not recorded.

Inference was run on one NVIDIA RTX 3090 at a resolution of 480 × 240. The average end-to-end runtime per test triplet, averaged over each test set, includes optical-flow estimation and tokenizer encoding and decoding: 0.978 s for WEB360, 0.475 s for 360VDS, and 0.908 s for FlowScape.

Response to Q3——More Qualitative Cases:

Thank you for this suggestion.

We have added additional qualitative examples to the revised manuscript and appendix. The examples cover diverse scenes and include both representative predictions and challenging cases. Each panel labels the blended input, ground truth, and PADiT prediction, with enlarged crops for selected regions. The blended input is shown for visualization; PADiT receives the two input frames separately.

Response to Q4——Failure Cases and Limitation:

Thank you for pointing out the need to discuss failure cases and the limitations of our method.

We acknowledge that although PADiT generally preserves the overall scene layout and major structures, its predictions may remain inaccurate in some regions. In particular, thin or rapidly moving structures, small objects.

These failure cases reflect limitations of the current method: flow-guided alignment depends on the accuracy of optical-flow estimation, while content newly revealed by motion or occlusion is difficult to infer reliably from only two input frames. We have added failure cases and a discussion of these limitations to the Appendix B and C. Overall, our method achieves favorable subjective quality across most scenes.

We again thank the reviewer for their careful, thorough, and constructive comments!

Thanks for the reply, but I don't see a clear discussion of limitation and motivation for tokenizer pretraining. The explanation of tokenizer pretraining makes sense but that does not address my issues about why it is necessary. And I can't find the added appendix B and C. I will keep my score.

AuthorsOpenReview ↗

Response to Reviewer y9wj

Thank you for your timely feedback. We sincerely apologize for the delayed upload of our revised manuscript, this delay arose from miscoordination within our team.

We have just uploaded the updated revised manuscript, which includes Appendix B and Appendix C.

Q1: Limitation and motivation for tokenizer pretraining.

R1: The motivation of tokenizer pretraining is to first build a compact latent representation capable of encoding and reconstructing panoramic frames, before optimizing the diffusion Transformer for frame interpolation. In the first stage, the tokenizer learns representations adapted to the appearance statistics of panoramic images, including scene structures and fine visual details. The diffusion Transformer then predicts target frames within this learned latent space, rather than modeling raw high-resolution pixels directly. This reduces the dimensionality of the prediction space and renders diffusion-based optimization more tractable. Second, pre-training the encoder and decoder prior to conditional diffusion training establishes a stable representation and reconstruction pathway. Accordingly, the tokenizer can be frozen during the initial DiT training phase, letting the DiT focus on learning the conditional distribution of latent representations for intermediate frames. Joint fine-tuning is subsequently performed to adapt both components together.

Tokenizer pre-training follows the common latent-space diffusion paradigm widely adopted in video frame interpolation. EDEN adopts a related strategy, while PADiT leverages its dedicated tokenizer architecture and panoramic-aware conditioning modules.

Nevertheless, tokenizer pre-training has its limitations. Since the tokenizer is optimized for single-frame encoding and reconstruction, its objective may not perfectly align with the requirements of conditional frame interpolation. The learned latent representations may attenuate subtle or high-frequency details, and information lost during encoding cannot be recovered by the diffusion Transformer. Moreover, tokenizer pre-training introduces an extra standalone training stage and additional computational overhead.

We also add these discussions of motivations and limitations to Appendix E.

Reviewer_5hdYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PADiT performs frame interpolation for equirectangular panoramic video. A Transformer tokenizer maps frames into a latent space, where an MT-DiT predicts the intermediate frame using angular positional encoding, multiscale feature fusion, flow-aligned attention with a confidence bias, and global motion conditioning. Training first learns the tokenizer, then trains the diffusion model with the tokenizer frozen, and finally jointly fine-tunes them with reconstruction and DINOv2 representation-alignment losses. The paper trains on WEB360 and evaluates on WEB360, 360VDS, and FlowScape, reporting perceptual metrics and runtime with two denoising steps.

Critical strengths
  1. The reported perceptual gains are consistent across datasets. In Table 1, WEB360 LPIPS/FloLPIPS decrease from the best baseline's 0.0154/0.0308 to 0.0141/0.0280, with PSNR reaching 37.646. FlowScape FloLPIPS decreases from 0.0268 to 0.0243. The agreement across datasets supports the effectiveness of the overall system.

  2. The experiments include both individual and cumulative component ablations: Tables 2–5 in Section 4.3 and Table 6 in Appendix A. Section 4.2 also states that baselines are retrained on the same panoramic data at the same resolution, and Table 1 includes optical flow and tokenizer execution in its timing. These are useful controls for evaluating the combined architecture.

Critical weaknesses
  1. The described implementation does not adequately explain the claimed spherical boundary handling. Applying Eq. (1)'s encoding to longitude as stated in Section 3.1.1 gives a sine-channel difference of 2sin(π/a_k) between the two longitude limits, where a_k = 10000^(2k/d_m). This is generally nonzero for k > 0: converting positions to angles does not itself make the encoding periodic. Section 3.1.2 claims that patchification preserves boundary adjacency, but Figure 2, Section 3.1.3, and Eqs. (2)–(3) do not specify circular indexing, boundary sampling, or explicit seam connections. Tables 2–6 report aggregate improvements without isolating seam behavior. The concern is not that the complete model must fail, but that its main geometric distinction from a generic DiT interpolator is not established by the description and evidence.

  2. The data protocol and sampling configuration leave the main results difficult to interpret. Section 4.1 describes 2,114 × 33 = 69,762 WEB360 triplets as training data, but does not specify the test video/triplet counts or source-video separation for the WEB360 results in Table 1. Temporal spacing for triplet construction is also unspecified. Separately, Figure 1(c) uses two denoising steps, but Section 3.2/Eq. (6), Section 4.1, and Appendix A do not provide the noise schedule, sampler, timesteps, or update rule; Table 1 omits the diffusion baselines' sampling budgets. These choices affect task difficulty, quality, and runtime. Their absence prevents a reliable reading of the claimed quality–speed advantage; it does not establish that train/test leakage occurred.

Questions for authors
  1. Does the implementation use Eq. (1)'s stated encoding for longitude? Please specify the boundary rules for patchification, warping, and attention. On a small set of existing test examples, could you report seam-region error or a longitude-shift consistency test? Explicit boundary handling supported by such a diagnostic would help resolve the geometric concern.

  2. What are the WEB360 training, validation, and test video/triplet counts, and are source videos disjoint? What frame spacings and interpolation times are used for each dataset? Do tokenizer pretraining, joint fine-tuning, and baseline training respect the same split?

  3. What noise schedule, sampler, two denoising timesteps, and update rule produced Table 1? Please distinguish diffusion time from interpolation time and list each diffusion baseline's step count. A verified video-disjoint split, reproducible sampling settings, and evidence for the claimed boundary treatment would be grounds to raise my recommendation to acceptance.

2 comments
AuthorsOpenReview ↗

Response to 5hdY Question 1: Seam Handling and Region-Wise Evaluation

We thank the reviewer for these concrete follow-up questions. We answer them point-by-point below:

Response to Q1_1: We fully agree that feeding the raw longitude ϕ∈[−π,π]\phi \in [-\pi,\pi] into the standard sinusoidal position encoding in Eq. (1) does not inherently enforce periodic continuity across the seam at ϕ=±π\phi=\pm\pi. This is a subtle but important distinction we failed to clarify in the original manuscript. For a longitude coordinate ϕ\phi, Eq. (1) uses channels of the form sin⁡(ϕ/ak)\sin(\phi/a_k) and cos⁡(ϕ/ak)\cos(\phi/a_k). Shifting the longitude by 2π2\pi changes the phase by 2π/ak2\pi/a_k, which is generally not an integer multiple of 2π2\pi.

Response to Q1_2: Our implementation handles the seam separately in the feature-processing stages. Patch indices wrap around along the longitude direction, so the leftmost and rightmost patches are treated as neighbors. During optical-flow warping on low-resolution feature maps, horizontal sampling coordinates are wrapped modulo the feature-map width WfW_f, while vertical sampling follows a non-periodic boundary rule. The attention module also explicitly connects tokens at the two longitude boundaries, allowing seam-adjacent features to interact.

Response to Q1_3: We evaluated the test predictions against the ground truth and report the mean per-sample PSNR for the full image, seam and non-seam regions, and three latitude bands. As shown in the table below. The seam region comprises the leftmost and rightmost 55\\% of the ERP width. The latitude bands are defined by ∣lat∣<30∘\lvert\mathrm{lat}\rvert<30^\circ, 30∘≤∣lat∣<60∘30^\circ\leq\lvert\mathrm{lat}\rvert<60^\circ, and ∣lat∣≥60∘\lvert\mathrm{lat}\rvert\geq60^\circ.These measurements provide a regional error analysis of PADiT alongside the implementation details of its seam handling. The ablation study further evaluates the contribution of each geometry module.

RegionWEB360 Dataset360VDS Dataset
PSNR (dB)PSNR (dB)
Full image37.6535.21
Seam region37.4735.34
Non-seam region37.7235.19
Equatorial band37.7635.59
Mid-latitude band37.5835.27
Polar band37.6334.81
AuthorsOpenReview ↗

Response to 5hdY Question 2 and 3

We thank the reviewer for this insightful and critical observation. We answer them one-by-one below:

Response to Q2:

We thank the reviewer for pointing out these missing implementation details.

The WEB360 contains 2,114 source videos. We split them by source video into 1,911 training videos and 203 test videos, with no source-video overlap. We construct 33 triplets from each video, yielding 63,063 training triplets and 6,699 test triplets. We did not use a separate validation set. The validation loader reused the test triplet list for progress monitoring only; its metrics were not used for checkpoint selection or hyperparameter tuning.

For WEB360 and 360VDS, each triplet consists of three consecutive frames in the prepared frame sequence: the first and third frames are the inputs, and the middle frame is the target. The input frames are therefore two frame indices apart, and the target is at normalized interpolation time t=0.5t=0.5. This specifies the spacing in frame indices, not a common interval in seconds. The latter depends on each video’s frame rate or extraction timestamps. For FlowScape, we use the middle-frame interpolation setting, t=0.5t=0.5. The synthetic sequences are sampled at 30 Hz; when using consecutive frames, the two input frames are separated by 2/302/30 seconds.

Tokenizer pretraining, joint fine-tuning, and the retrained baselines use the same WEB360 training split. We will add these implementation details to Appendix D and release our complete code publicly for reproducibility.

Response to Q3:

Thank you for requesting these reproducibility details. We address each point below.

Q3_1. Noise schedule and sampler.

The PADiT uses a linear-transport formulation with velocity prediction and a two-step Euler ODE sampler. It does not use a DDIM beta noise schedule. The solver time grid is 0,0.75,10, 0.75, 1, giving two Euler updates over [0,0.75][0,0.75] and [0.75,1][0.75,1].

Q3_2. Update rule.

At solver time τi\tau_i, the state is updated as

zτi+1=zτi+(τi+1−τi)vθ(zτi,τi,c),z_{\tau_{i+1}} = z_{\tau_i} + (\tau_{i+1}-\tau_i)v_\theta(z_{\tau_i},\tau_i,c),

where vθv_\theta is the predicted velocity and cc denotes the conditioning inputs. Specifically, the two updates use (τ0,τ1)=(0,0.75)(\tau_0,\tau_1)=(0,0.75) and (τ1,τ2)=(0.75,1)(\tau_1,\tau_2)=(0.75,1).

Q3_3. Diffusion time versus interpolation time.

The interpolation time t=0.5t=0.5 specifies that the target frame lies midway between the two input frames. The solver time τ\tau instead parameterizes integration of the transport ODE. Thus, the solver nodes 0,0.75,10, 0.75, 1 are distinct from the interpolation time.

Q3_4. Sampling budgets for diffusion baselines.

EDEN is sampled with a 2-step Euler ODE solver, CBBD with 50 steps, and LDMVFI with 200-step DDIM.

Q3_5.Video-disjoint split and boundary evidence.

WEB360 is split by source video into 1,911 training videos (63,063 triplets) and 203 test videos (6,699 triplets), with no source-video overlap. There is no separate validation split; the test list was used by the validation loader for progress monitoring only, and its metrics were not used for checkpoint selection or hyperparameter tuning. We report seam-region and latitude-stratified PSNR for WEB360 and 360VDS in our response to the boundary-handling question. These regional measurements provide diagnostic evidence of PADiT’s performance in those regions, alongside the implementation details of seam handling.

We thank the reviewer again for their careful, thorough, and constructive comments!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.