acceptodds
Under review as a conference paper at ICLR 2027

Safe4D: Causal Video Protection against Unauthorized 4D Reconstruction

Abstract

Recent feed-forward geometry models recover cameras, depth and dynamic point clouds from ordinary video, either offline from a complete clip or online as frames arrive, turning a video into a renderable 4D asset within minutes. However, this capability raises serious concerns about privacy and intellectual property, since anyone who records a published video can recover the shape and motion of the people, products and places it shows. Existing protection methods perturb images or complete videos before release, but live broadcasts and video calls are released causally: each frame is published as soon as it is captured and cannot be revised, while viewers can record the stream and reconstruct it. Protecting video against 4D reconstruction under such causal release remains unexplored and raises three challenges: (i) temporal detachment: a perturbation fixed in image coordinates drifts off the moving content it protects and is averaged out by motion-aligned purification; (ii) purification uncertainty: the attacker may reconstruct from the raw or the purified video, and a perturbation tuned to one input leaves the other unprotected; and (iii) irrevocable release: each frame is transmitted as an 8-bit image as soon as it is protected, so an update weakened by rounding cannot be corrected later. To address these challenges, we propose Safe4D, the first video protection method against 4D reconstruction under causal release. Safe4D transports the perturbation committed at the previous frame along the estimated scene motion, so that the protection stays attached to the scene points it perturbs. Moreover, we introduce a purification-aware objective on a frozen streaming surrogate, which maximizes the weaker of the depth discrepancies on the direct and the temporally averaged stream, and a verified commit that accepts a sparse 8-bit update only if the quantized frame to be released improves this objective. Empirically, quantitative and qualitative results on four dynamic benchmarks show that Safe4D effectively disrupts the geometry recovered by streaming 4D reconstruction and remains robust under both causal and non-causal temporal purification.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
4D Reconstruction, Video Protection

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_9Df6OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Previous:

Safe4D studies causal video perturbations against unauthorized 4D reconstruction. The evidence mainly supports degradation of predicted depth and geometry, with transfer to some unseen models. Robustness to compression and purification, runtime, and effects on final rendering remain limitations.

Update:

This paper propose to use the protective effect of causal video perturbations on feed-forward reconstruction models, with evidence primarily drawn from the observed increase in depth and geometric errors. Experiments reveal that the proposed protective effect can generalize to some OOD models. But, there are still some limitations in video compression and rendering quality, etc.

Critical strengths

Previous:

The problem formulation and causal setting are valuable. The added transfer experiments strengthen the evidence: depth error increases on 61/62 sequences for VGGT and 62/62 for STream3R without receiver-specific tuning. The temporal-filtering results also support robustness to the tested filters.

Update:

It is a pioneering work on protecting video from 4D reconstruction, and extensive works about feed-forward 4D reconstruction models are compared in the appnedix. This experiement reinforces the evidence for cross-model transferability.

Critical weaknesses

Previous:

Transfer remains model-dependent, with inconsistent results on CUT3R. H.264 compression substantially weakens protection, and DiffPure nearly removes the effect on the single tested clip, although resizing alone also reduces it. The viewing test does not support an unconditional imperceptibility claim. Runtime remains high, and the added all-camera 4DGS experiment does not show consistent rendering degradation. Geometry-level disruption therefore does not yet establish reliable end-to-end protection.

Update:

When using direct input, depth error increased, with the average change being slightly negative. Consequently, transferability of the proposed method is highly dependent on the selected model.The receiver and agent models share the same encoder, but CUT3R does not. Therefore, current experiments do not treat the encoder as a variable.

Questions for authors
  1. Can the author report the full distribution of transfer results (not just the mean) and analyze why transfer fails on some sequences?
  2. Is there a path to reduce the per-frame cost by 2–3 orders of magnitude (e.g., an amortized perturbation generator)?
  3. Could you add a small human annotation to prove the experiental insights?
10 comments
AuthorsOpenReview ↗

Response to Reviewer 9Df6 (Part 1/5)

Dear Reviewer 9Df6,

Thanks for your constructive comments! We respond to each of your points below. Please feel free to let us know if further details or explanations are helpful.

In all tables of this response, ↑ (↓) after a metric means that a higher (lower) value is better from the protector's side. For example, a higher depth error means stronger protection.

W1 & Q1: What is the full distribution of the transfer results? Why does transfer sometimes fail?

Response 1: Thank you for the detailed comments. We answer the three parts in turn.

(1) Full distribution. The submission reports every sequence for CUT3R [c], a streaming model with a different architecture and training data, in Appendix G (Tables 20–23 in the revised manuscript). Following your suggestion, we add two more unseen models, VGGT [a] (offline) and STream3R [b] (streaming), report every sequence for them as well (Table 37 in the revised manuscript), and summarize the three models in Table C1 (Table 5 in the revised manuscript). Safe4D is optimized against StreamVGGT only, and the three models receive the same protected videos without retuning. On the released videos, the depth error increases on 61/62 sequences for VGGT and 62/62 for STream3R, and on 34/62 for CUT3R.

(2) Why transfer sometimes fails. The failures concentrate on CUT3R. VGGT and STream3R are related to StreamVGGT, and transfer to them is consistent. CUT3R has a different architecture and training data: the perturbation still changes its prediction, but without a consistent direction. On the 23 Sintel sequences, the median change in its depth error is +1.52 × 10⁻³, and the middle half of the sequences lies between −23.8 × 10⁻³ and +36.1 × 10⁻³.

Table C1: Depth transfer on all 62 depth sequences. ΔAbsRel is the absolute relative depth error on the protected video minus that on the clean video under the same processing (a positive value means worse reconstruction). Each cell gives the mean ΔAbsRel over the sequences in units of 10⁻³ and, in brackets, the number of sequences with a higher error. "Forward two-frame average" means that the attacker averages every frame with the next one.

Unseen modelDirect ΔAbsRel ×10⁻³ (↑)Forward two-frame average ΔAbsRel ×10⁻³ (↑)
VGGT+214 (61/62)+207 (60/62)
STream3R+222 (62/62)+192 (59/62)
CUT3R−0.77 (34/62)+4.30 (37/62)

References

[a] VGGT: Visual Geometry Grounded Transformer.

[b] STream3R: Scalable Sequential 3D Reconstruction with Causal Transformer.

[c] Continuous 3D Perception Model with Persistent State.

AuthorsOpenReview ↗

Response to Reviewer 9Df6 (Part 2/5)

Q2: How much protection survives temporal smoothing or DiffPure? Does the +1.94 dB result imply that the attacker can bypass the protection?

Response 2: Thank you for the careful reading, and we apologize for the confusion about the result. We answer the two parts in turn.

(1) Temporal smoothing and DiffPure.

Experimental setup: On the complete benchmarks (Sintel, Bonn, KITTI), three temporal filters are applied to the protected videos: a two-frame average and two motion-aligned three-frame medians (Table 2 in the manuscript). Following your suggestion, we conduct two more experiments on one 50-frame Sintel clip (alley_1): a centered five-frame average, and frame-by-frame DiffPure [d], which adds noise to each frame and removes it with a diffusion model (noise timestep 150). DiffPure runs at 256 × 256, so we also report a resize-only control that applies the same down- and up-sampling without diffusion.

Results: Under the three temporal filters, Safe4D keeps 91.3–108.3% of its depth-error increase on the three datasets. On alley_1 (Table C2; Tables 33 and 35 in the revised manuscript), the five-frame average keeps 70.7%.

Table C2: Attacker-side processing on all 50 frames of alley_1 (AbsRel, the absolute relative depth error of StreamVGGT). Bracketed values indicate changes relative to the clean video under the same processing, computed from unrounded scores.

ProcessingClean AbsRel (↑)Protected AbsRel (↑)
None (direct)0.1730.844 (+0.671)
Centered five-frame average0.2120.686 (+0.474)
Resize only0.2400.286 (+0.0462)
DiffPure0.2740.275 (+0.000186)

(2) The +1.94 dB result. This number comes from a supplementary experiment in the appendix (Table 25 in the submission, Table 27 in the revised manuscript). It reconstructs one scene with 4D Gaussian splatting, with one protected camera out of 20, and it uses the earlier Same-grid variant (which copies the previous perturbation to the same pixels) instead of the final Safe4D. In that run, the answer is yes: when the attacker skips the filtering, the rendered held-out views are not degraded. Their PSNR (peak signal-to-noise ratio; higher means better rendering) is 1.94 dB higher than with the clean input, and after two-frame averaging it is 2.23 dB lower. For depth, which is what Safe4D optimizes, skipping the filter does not help the attacker: without any filtering, the depth error is already 1.96–3.66× that of the clean video (Table 1).

References

[d] Diffusion Models for Adversarial Purification.

AuthorsOpenReview ↗

Response to Reviewer 9Df6 (Part 3/5)

W2 & Q3: Could an amortized generator reduce the per-frame cost by two to three orders of magnitude?

Response 3: Thank you for the constructive suggestion. Following your suggestion, we conduct a preliminary experiment that trains a small network to output the perturbation in one forward pass, instead of optimizing every frame iteratively.

Experimental setup: The generator is a small convolutional network (three 3 × 3 convolution layers, about 11k parameters). It takes the current frame as input and is trained to reproduce the perturbation that iterative Safe4D produces for that frame. It is trained for 300 updates on the first 16 frames of one Sintel clip (alley_2) and tested on all 50 frames of another clip (alley_1) that it has never seen.

Results: The results are reported in Table C3 (Table 34 in the revised manuscript). The generator produces a perturbation in 0.280 ms per frame, but this perturbation keeps only 0.46% of the depth-error increase of iterative Safe4D The network here is very small and is trained for only 300 updates on a single clip. Training and evaluating a generator on the full datasets takes longer time, and we will report those results in the final version.

Table C3: One-pass generator vs. iterative Safe4D on the held-out clip alley_1. The two timings are measured differently and are not an end-to-end comparison: 0.280 ms covers network inference, clipping and rounding on one H200 GPU, and 291 s is the median over the 70 benchmark videos.

MethodTime per frame (↓)Protected AbsRel (↑)ΔAbsRel (↑)Protection kept (↑)
Iterative Safe4D291 s0.844+0.671100%
One-pass generator0.280 ms0.176+0.00310.46%
Reviewer_rc58OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Safe4D, a causal video protection framework designed to prevent unauthorized 4D reconstruction from sequentially released video streams. Unlike conventional image or video protection methods that assume access to complete content before release, Safe4D generates adversarial perturbations using only current and past observations, while previously published frames remain immutable. The proposed framework consists of three main components: motion-transported initialization to maintain perturbation consistency across moving scene content, a purification-aware objective that considers both direct and temporally purified inputs, and a sparse verified-commit mechanism that ensures updates improve the objective after 8-bit quantization. Experiments on MPI Sintel, Bonn RGB-D, KITTI, and TUM RGB-D demonstrate substantial degradation of geometric reconstruction on the white-box surrogate StreamVGGT. Additional evaluations investigate robustness against temporal purification, transferability to the unseen CUT3R model, and downstream 4D reconstruction.

Critical strengths
  1. The paper investigates an important and relatively unexplored problem of protecting video streams against unauthorized 4D reconstruction under causal-release constraints. Unlike conventional offline protection methods, Safe4D must generate perturbations without accessing future frames or modifying previously published content. This formulation introduces meaningful challenges and provides a valuable new perspective on the security of streaming visual reconstruction systems.

  2. Safe4D combines motion-transported initialization, purification-aware depth optimization, and verified 8-bit updates (Eqs. (4)–(7)). These components directly address temporal detachment, purification uncertainty, and irreversible frame publication, respectively. The resulting framework provides a coherent and task-specific solution rather than simply applying conventional adversarial perturbations to video frames.

  3. The paper evaluates Safe4D on four dynamic benchmarks comprising 70 sequences in Table 1 and investigates robustness under causal and non-causal temporal purification in Table 2. The results demonstrate substantial degradation of white-box geometric reconstruction while maintaining protection under different purification strategies. Additional analyses of cross-model transfer, native 3D geometry, and downstream 4D reconstruction further characterize the effectiveness and limitations of the proposed framework.

Critical weaknesses
  1. The proposed optimization requires multiple surrogate evaluations per frame, with 64 calls under the default configuration. The appendix reports that processing a 300-frame sequence takes approximately 9.35 hours, with per-frame processing time exceeding 200 seconds toward the end of the sequence. Although the method satisfies the causal information constraint, its current computational requirements are incompatible with practical real-time streaming. The authors should more clearly distinguish causal operation from real-time feasibility and discuss potential approaches for reducing computational and memory costs.

  2. The downstream reconstruction results vary substantially across different configurations (Table 25). For example, under calibrated cameras and SIFT initialization, protection reduces PSNR by 2.23 dB after purification but instead improves PSNR by 1.94 dB without purification. Moreover, under VGGT-predicted geometry, the post-purification PSNR degradation decreases to only 0.0972 dB. These results suggest that protection effectiveness is highly dependent on the reconstruction pipeline. Since these experiments also use an earlier same-grid variant rather than the final Safe4D framework, broader downstream evaluations of the complete method would strengthen the practical claims.

  3. Although Safe4D demonstrates strong white-box protection on StreamVGGT, its effectiveness does not consistently transfer to the unseen CUT3R model in Table 15. Across all 70 benchmark sequences, the protected videos fail to consistently increase reconstruction errors, with some metrics even improving. This contrasts with the favorable results on three selected Sintel sequences (Table 3). Since unauthorized reconstructors may employ different architectures, the limited cross-model transferability raises concerns about the generalizability of the proposed protection. If there is some further analysis of the conditions under which transfer succeeds or fails, it would strengthen the paper.

Questions for authors
  1. Safe4D primarily optimizes discrepancies in predicted depth structure, whereas downstream 4D reconstruction also depends on camera poses and temporal motion consistency. Could the authors explain why depth discrepancy alone is an appropriate surrogate for disrupting complete 4D reconstruction? Would incorporating pose or motion-related objectives potentially improve protection effectiveness?

  2. The purification-aware objective uses a causal two-frame average to approximate potential attacker-side preprocessing. What motivates this particular choice, and how sensitive is the proposed method to the temporal window size or the type of purification used during optimization?

  3. The proposed initialization relies on estimated optical flow to propagate perturbations across consecutive frames. How does the accuracy of the flow estimation affect protection effectiveness, particularly under large motions, occlusions, or unreliable correspondences?

9 comments
AuthorsOpenReview ↗

Response to Reviewer rc58 (Part 1/5)

Dear Reviewer rc58,

Thanks for your valuable comments! We address the weaknesses and questions below. Please kindly let us know if you have further concerns.

In all tables of this response, ↑ (↓) after a metric means that a higher (lower) value is better from the protector's side. For example, a higher depth error means stronger protection.

W1: Can Safe4D keep up with a live stream? How could its runtime and memory use be reduced?

Response 1: Thank you for raising this practical point, and we apologize for not reporting the per-frame cost in the main text. We acknowledge that the current implementation cannot keep up with a live stream: protecting one frame takes a median of 291 s. Safe4D is causal, i.e., it uses only the current and past frames, but being causal does not make it fast.

Where the cost comes from:

  • Runtime: For every frame, Safe4D runs 10 refinement steps. Each step needs six calls of the reconstruction model, and each frame needs four more calls, which gives 64 calls per frame, plus optical flow.
  • Memory: The reconstruction model keeps the features of all past frames, so the memory and the time per frame grow with the length of the stream.

How it could be reduced:

  • Fewer refinement steps, which reduce the number of model calls.
  • A fixed-length history instead of the full history, which bounds the memory.
  • A one-pass generator that outputs the perturbation directly. Following this direction, we conduct a preliminary experiment: the generation time drops to 0.280 ms per frame, but the generated perturbation keeps only 0.46% of the depth-error increase of Safe4D (Table 34 in the revised manuscript).

Therefore, the open problem is to reduce the cost while keeping the protection, and we state it as a limitation in the revised manuscript. The measurements are reported in Table B1 (Section 5.6 and Table 30 in the revised manuscript).

Table B1: Cost of protecting one frame. Same-grid is our baseline that copies the previous perturbation to the same pixels instead of moving it with the scene. Its 300-frame video comes from an earlier run at a higher resolution, and it shows how the cost grows with the length of the stream.

SettingMetricMeasurement
Safe4D, 70 benchmark videosTime per frame (↓)291 s (median; the runtime of each video divided by its number of frames)
Safe4DModel calls per frame (↓)64 calls of the reconstruction model, plus optical flow
Same-grid, one 300-frame videoTime per frame (↓)112 s on average, rising from 26.0 s (frames 8–49) to 206 s (frames 250–299)
Same-grid, the same videoMemory (↓)13.5 GiB peak GPU memory, and 229 GiB of history kept in CPU memory at the last frame
AuthorsOpenReview ↗

Response to Reviewer rc58 (Part 2/5)

W2: How much does the downstream result depend on the reconstruction pipeline?

Response 2: Thank you for the insightful comment, and we apologize for the confusion caused by the result in the appendix. In short, the depth damage carries over to VGGT and STream3R, the results on CUT3R are mixed for both depth and 3D shape, and the effect on rendered views depends on the pipeline in our only test.

Why these experiments: An attacker can turn a video into a 4D asset with different kinds of pipelines. To avoid relying on a single pipeline, we give the same protected videos, without retuning, to one representative of each kind. Each kind also tests a different stage of the reconstruction:

  • Feed-forward models related to StreamVGGT, the model that Safe4D is optimized against (VGGT [a], which is offline, and STream3R [b], which is streaming): they output depth directly, so we measure the depth error.
  • CUT3R [c], a streaming model with a different architecture and training data: we compare its output point clouds with the ground-truth 3D shape using the Chamfer distance (the average distance from each point to its nearest point in the other point cloud).
  • 4D Gaussian splatting (4DGS) [d], an optimization-based pipeline that produces a renderable asset: we measure the quality of rendered held-out views with PSNR (peak signal-to-noise ratio; higher means better rendering).

Results:

  • Depth: The depth error increases on 61/62 sequences for VGGT and on 62/62 for STream3R, while the result is mixed for CUT3R (Table B3 in Response 3).
  • 3D shape: On three 16-frame clips, the point clouds of CUT3R move farther from the ground truth with the final Safe4D, both on the released video and after the attacker averages every frame with the next one, which we call the forward two-frame average (Table B2, Table 4 in the manuscript). We then extend this evaluation to all 23 Sintel sequences. On the released video, the point-cloud error of CUT3R increases on 11 of the 23 sequences. Therefore, the three clips are not representative, and the 3D-shape result on CUT3R is mixed, like its depth result.
  • Rendered views: We have one experiment of this kind, in the appendix (Table 25 in the submission, Table 27 in the revised manuscript). Its setting is demanding: besides the protected video, the attacker owns 19 unprotected cameras of the same scene, so only one of the 20 training cameras is protected, and it uses the earlier Same-grid variant. With calibrated cameras, the protected input gives a PSNR that is 1.94 dB higher than the clean input when it is used directly, and 2.23 dB lower after two-frame averaging. With VGGT-predicted geometry, the drop after averaging is 0.0972 dB.

Table B2: Increase in the Chamfer distance (ΔCD) of the CUT3R point clouds caused by Safe4D, in % of the value for the clean video under the same processing (three 16-frame Sintel clips, S1–S3 in the manuscript).

Input to CUT3RS1 ΔCD (↑)S2 ΔCD (↑)S3 ΔCD (↑)
Released video (direct)+13.2%+6.50%+9.15%
After forward two-frame average+11.2%+5.77%+10.1%

References

[a] VGGT: Visual Geometry Grounded Transformer.

[b] STream3R: Scalable Sequential 3D Reconstruction with Causal Transformer.

[c] Continuous 3D Perception Model with Persistent State.

[d] 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering.

AuthorsOpenReview ↗

Response to Reviewer rc58 (Part 3/5)

W3: How consistently does the protection transfer beyond StreamVGGT?

Response 3: Thank you for the constructive comment. The submission reports the transfer to CUT3R on the complete benchmarks in Appendix G. Following your suggestion, we add two more unseen models, VGGT and STream3R, and summarize the three models on all 62 depth sequences in the main text.

Experimental setup: Safe4D is optimized against StreamVGGT only. We give the same protected videos, without retuning, to three models that the protector never queries:

  • VGGT [a]: an offline model that processes the whole sequence at once.
  • STream3R [b]: a streaming model with a causal transformer.
  • CUT3R [c]: a streaming model with a different architecture and training data.

Each model receives the released video as it is ("direct"), or after the attacker averages every frame with the next one ("forward two-frame average"). For each sequence we report ΔAbsRel, i.e., the absolute relative depth error on the protected video minus that on the clean video under the same processing. A positive value means worse reconstruction.

Results: The results are reported in Table B3 (Table 5 in the revised manuscript), and the per-sequence results in Tables 20–23 and 37. The protection transfers consistently to VGGT and STream3R: the depth error increases on 61/62 and 62/62 sequences. On CUT3R, the error increases on 34/62 sequences, and the mean change is close to zero. The eight TUM sequences are scored on camera pose and reported separately (Tables 23 and 37).

Why CUT3R differs: Safe4D is optimized against StreamVGGT only. VGGT and STream3R are related to StreamVGGT, whereas CUT3R has a different architecture and training data. On CUT3R, the perturbation still changes the prediction, but without a consistent direction. Optimizing against several models from different families is the direction we will study.

Table B3: Depth transfer on all 62 depth sequences (Sintel 23, Bonn 26, KITTI 13). Each cell gives the mean ΔAbsRel over the sequences in units of 10⁻³ and, in brackets, the number of sequences with a higher error.

Unseen modelDirect ΔAbsRel ×10⁻³ (↑)Forward two-frame average ΔAbsRel ×10⁻³ (↑)
VGGT+214 (61/62)+207 (60/62)
STream3R+222 (62/62)+192 (59/62)
CUT3R−0.77 (34/62)+4.30 (37/62)

References

[a] VGGT: Visual Geometry Grounded Transformer.

[b] STream3R: Scalable Sequential 3D Reconstruction with Causal Transformer.

[c] Continuous 3D Perception Model with Persistent State.

Reviewer_PDPAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses a practical problem: in a live broadcast or video call, each frame is released as soon as it is captured and can never be revised. However, viewers can record the whole stream and use current 4D reconstruction models to recover the people, objects, and scene as 3D content that can be viewed from new angles (4D because it changes over time). The authors call protection under this constraint "causal video protection": the current frame can only be modified based on frames that have already been released.

Their method, Safe4D, does three things:

  1. It makes the noise follow the objects in the scene. If the noise stays fixed at the same screen location, it drifts away from the object it is meant to protect as soon as the object moves. An attacker only needs to align neighboring frames by motion and average them, and the noise cancels out. The authors therefore use optical flow (estimating where each pixel moves from the previous frame) to carry the previous frame's noise to the current frame.
  2. It considers two kinds of attacker at once. One uses the video directly; the other first applies temporal smoothing. The authors try to make protection effective in both cases, and in particular to keep the weaker of the two from becoming too weak.
  3. It changes only a small number of pixels per frame. After the change, it re-checks the actually released 8-bit image; if the result is not better, the change is not kept.

The main model is StreamVGGT, transfer is tested on the unseen CUT3R. Several temporal smoothers are also evaluated. Results show that depth error roughly doubles, and that most of the effect survives one motion-aligned smoother.

Critical strengths
  1. The core idea that the noise must follow the object is correct and clearly explained. The authors give a simple derivation: after two frames are aligned by motion and averaged, noise that does not follow the object cancels out, while noise that does follow it survives.

  2. The experiments and reporting are honest. The authors compare not only against external methods but also against their own simplest variant (copying the previous frame's noise to the same pixels, without optical flow).

Critical weaknesses
  1. In the main setting, the method is barely better than the simplest baseline. In Table 1, the simple "copy the previous noise" variant (Same-grid) is almost identical to Safe4D: Sintel 0.476 vs. 0.478, Bonn 0.219 vs. 0.219, KITTI 0.324 vs. 0.327, TUM 0.108 vs. 0.0996. In other words, the entire "follow the object" design makes almost no difference in the main white-box setting. It is clearly better only when the attacker uses motion-aligned temporal smoothing.
Questions for authors
  1. How much protection remains after video compression? The experiments publish frames as lossless PNG, but live streaming always involves compression.
  2. Is the noise visible to the eye? Can you provide PSNR/LPIPS or magnified comparisons? The paper says "the video looks the same" but gives no measurement.
  3. How long does each frame take with protection enabled? Can it meet real-time streaming?
10 comments
AuthorsOpenReview ↗

Response to Reviewer PDPA (Part 1/3)

Dear Reviewer PDPA,

Thanks for your valuable reviews! We address your concerns as follows. Please let us know if anything remains unclear.

In all tables of this response, ↑ (↓) after a metric means that a higher (lower) value is better from the protector's side. For example, a higher depth error means stronger protection.

W1: The white-box scores of Safe4D and the Same-grid baseline in Table 1 are close. When does motion transport help?

Response 1: Thank you for the valuable comment, and we apologize for any confusion.

Why the scores in Table 1 are close: Same-grid is the baseline that copies the previous perturbation to the same pixel positions, while Safe4D moves it along the scene motion (motion transport). Everything else is identical. In Table 1, the attacker reconstructs the released video directly. Both methods are optimized against the same model with the same objective, so their depth errors are close. On KITTI, AbsRel (the absolute relative depth error; a higher error means stronger protection) is 0.324 for Same-grid and 0.327 for Safe4D.

Why they differ after filtering: Before reconstruction, an attacker can align neighboring frames by their motion and take the median, which removes whatever does not agree across frames. A perturbation that stays at fixed pixels drifts off the moving content and is filtered out, whereas a perturbation that moves with the content survives.

Results: The quantitative results are reported in Table A1 (the "Flow-median3" rows of Table 2 in the manuscript), and the qualitative results are shown in Figure 3. After the motion-aligned median filter, Safe4D keeps 91.3% of its depth-error increase on KITTI (the increase after filtering divided by the increase without filtering), whereas Same-grid keeps 54.2%. The remaining increase of Safe4D is 72.2% larger than that of Same-grid on KITTI and 106% larger on TUM. On Bonn, both methods increase the depth error on all 26 sequences after this filter, and they are within 1.60% of each other, so motion transport brings no additional gain on this dataset. In Figure 3 (a KITTI sequence after this filter), the point cloud of Same-grid stays close to the clean one (AbsRel 0.123 → 0.143), while that of Safe4D is visibly distorted (AbsRel 0.423). Therefore, close scores in Table 1 do not mean equal protection once the attacker filters the video.

Table A1: Increase in reconstruction error after the motion-aligned three-frame median filter (protected minus clean, both filtered in the same way). ΔAbsRel: increase in depth error. ΔATE: increase in camera trajectory error (×10⁻² m). Bracketed values indicate changes relative to Same-grid. Best results are highlighted in bold.

MethodSintel ΔAbsRel (↑)Bonn ΔAbsRel (↑)KITTI ΔAbsRel (↑)TUM ΔATE (↑)
Same-grid0.2050.1580.08470.482
Safe4D (Ours)0.235 (+14.6%)0.155 (−1.60%)0.146 (+72.2%)0.993 (+106%)
AuthorsOpenReview ↗

Response to Reviewer PDPA (Part 2/3)

Q1: How much protection survives video compression?

Response 2: Thank you for this important question. 4D reconstruction (Sintel, Bonn, KITTI and TUM) benchmarks are not built for live streaming: they provide losslessly stored frames, and their standard protocols do not include a codec.

Experimental setup: We use all 23 Sintel sequences (1,064 frames). The clean and the protected videos are encoded and decoded with the same settings (libx264, constant rate factor (CRF) 23, which is the default quality level of this encoder, medium preset, YUV420p, 30 fps). Both are then reconstructed by StreamVGGT [a], the model that Safe4D is optimized against. For each sequence, we compute ΔAbsRel, i.e., the depth error of the protected video minus that of the clean video under the same processing.

Results: The results are reported in Table A2 (Table 33 in the revised manuscript). On average over the sequences, the depth-error increase drops from +0.244 to +0.0170 after compression, i.e., 6.98% of it remains. The protected video still has a higher depth error than the clean video on 14 of the 23 sequences.

Table A2: Depth protection before and after H.264 compression on all 23 Sintel sequences (StreamVGGT). ΔAbsRel is computed for each sequence and then summarized over the 23 sequences.

Statistic over the 23 sequencesOriginal PNGH.264, CRF 23
Mean ΔAbsRel (↑)+0.244+0.0170
Median ΔAbsRel (↑)+0.203+0.00658
Mean ΔAbsRel, weighted by valid pixels (↑)+0.249+0.0246
Sequences with a higher error (↑)22/2314/23

References

[a] Streaming 4D Visual Geometry Transformer.

AuthorsOpenReview ↗

Response to Reviewer PDPA (Part 3/3)

Q2: Is the perturbation visible in the released video?

Response 3: Thanks for pointing this out, and we apologize for not reporting visibility measurements in the main text. Following your comment, we conduct a quantitative evaluation with two standard image-quality metrics.

Experimental setup: On two 50-frame Sintel clips (alley_1 and market_5), we compare every protected frame with its clean frame using PSNR (peak signal-to-noise ratio; higher means closer to the clean frame) and LPIPS [b] (a learned perceptual distance; lower means closer).

Results: The results are reported in Table A3 (Table 31 in the revised manuscript). On both clips, the PSNR is about 40 dB and the LPIPS is below 0.04, i.e., the pixel-level change caused by the protection is small.

Table A3: Quality of the released frames, averaged over all 50 frames of each clip.

ClipPSNR in dB (↑)LPIPS (↓)
alley_139.60.0279
market_541.20.0396

Q3: How long does the protection take per frame, and can it meet real-time streaming?

Response 4: Thank you for raising this practical concern. We acknowledge that the current proposed method may not be able to meet the real-time streaming: protecting one frame takes a median of 291 s.

Why it is slow: For every frame, Safe4D runs 10 refinement steps. Each step needs six calls of the reconstruction model to score and update the perturbation, and each frame needs four more calls, which gives 64 calls per frame, plus optical flow.

In this first study, we focus on how much protection can be achieved when each frame must be released before the future frames exist. Reducing the time per frame while keeping the protection is our next goal. The cost is reported in Table A4 (Section 5.6 and Table 30 in the revised manuscript).

Table A4: Cost of protecting one frame with Safe4D.

MetricMeasurement
Time per frame (↓)291 s (median over the 70 benchmark videos)
Model calls per frame (↓)64 calls of the reconstruction model (10 refinement steps), plus optical flow

References

[b] The Unreasonable Effectiveness of Deep Features as a Perceptual Metric.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers,

We appreciate all the reviewers for their insightful and constructive reviews of our manuscript. We are glad that the reviewers found that:

  • The problem of protecting a video while each frame is being released is novel (Reviewer rc58 and 9Df6).
  • The idea that a perturbation should follow the moving object it protects is well motivated (Reviewer PDPA).

Based on all the reviews, here we provide a general response to the concerns raised by multiple reviewers. The individual responses are commented on below each review.

  • Regarding questions about transfer and purification, we have addressed the concerns as follows:
    • We add two more unseen models, VGGT and STream3R, to the transfer evaluation on all 62 depth sequences. The depth error increases on 61/62 sequences for VGGT, 62/62 for STream3R and 34/62 for CUT3R. We also add a point-cloud evaluation of the three models on all 23 Sintel sequences, where the error increases on 19/23, 21/23 and 11/23 sequences. (For Reviewer rc58 and 9Df6)
    • We report robustness to temporal filtering on three datasets. Under the three filters in Table 2, Safe4D keeps 91.3–108.3% of its depth-error increase. Under the motion-aligned median filter on KITTI, Safe4D keeps 91.3%, whereas Same-grid (the baseline that copies the perturbation to the same pixels instead of moving it with the scene) keeps 54.2%. (For Reviewer PDPA, rc58 and 9Df6)
    • We add an H.264 compression experiment on all 23 Sintel sequences (Table 33 in the revised manuscript). (For Reviewer PDPA and 9Df6)
    • We add more purification experiments on one 50-frame clip: a centered five-frame average and frame-by-frame DiffPure with a resize-only control (Tables 33 and 35 in the revised manuscript). (For Reviewer 9Df6)
  • Regarding questions about practical use, we have addressed the concerns as follows:
    • We add visibility measurements with two standard image-quality metrics, PSNR and LPIPS, on two clips. (For Reviewer PDPA and 9Df6)
    • We add a small viewing study with one viewer on all 23 Sintel scenes (Appendix P in the revised manuscript). (For Reviewer 9Df6)
    • We add a runtime analysis of the current implementation and a preliminary experiment with a one-pass generator (Section 5.6, Tables 30 and 34 in the revised manuscript). (For all reviewers)
  • Regarding questions about design choices, we have addressed the concerns as follows:
    • We add an ablation of the geometry and motion objectives, a sensitivity study of the filter used during optimization, and an analysis of optical-flow reliability. (For Reviewer rc58)

We have also uploaded a revised version of our submission according to the suggestions provided by the reviewers, with major changes highlighted in blue. One additional experiment, rendering with the final Safe4D, is still running, and we will update our responses and the manuscript as soon as the results are available. We look forward to further feedback and discussion. Please feel free to let us know if further details or explanations are helpful.

Once again, we thank all reviewers for their thoughtful feedback and valuable guidance in improving our work.

Best regards,

Submission 20938 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.