- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
Building on EgoX, Ego-Forge generates a first-person video from a third-person video + head trajectory, using a pre-trained video model. Instead of relying on manual captioning Ego-Forge introduces a learned captioner “R”. It also claims that EgoXs geometry guided attention can be dropped when enough training data is available.
Critical strengths
The problem is interesting and relevant. Being caption-free at inference is a huge advantage.
Critical weaknesses
My main concern is that the paper leaves out a lot of crucial information of the architecture. The authors should add an architecture description of “R” - in its current state it is not possible to re-implement R as stated. For example, in Eq. 3 Q^{exo} and Q^{ego} have their separate attention calls while in Figure 3 they have the same (?) projections. Which is it? Where is the “value” part, it is missing in both Eq3 and Fig3. How is fed into R?
Figure 2 and Figure 3 are way too small. In general, Figure 3 is difficult to read, the two colors for K^{exo} and K^{ego} are almost the same. The and should also get different colors to not make them look similar to the mask. What positional embedding was used?
Some claims need to be softened, i.e.
- The paper states that LL053-055 “[EgoX] … can propagate reconstruction error ...“ but this is also true for Ego-Forge - it uses a depth-reprojected prior which comes with its own bias. Also crucially, this is not ablated in any of the experiments.
- The paper states (in its title) that GGA can be dropped (when training is scaled) - but they do not show any evidence for that! No ablation varies data size to measure this.
Questions for authors
Is the room-camera part of the canvas noised? In Figure 1 it looks like only the egocentric part is noised and the exo-part is kept clean, but in 3.1 it seems that both and are noised.
Figure 7: what are the differences in “Rendering” between EgoX and Ego-Forge? Should that not be the same operation in both pipelines? Why is it so much slower in EgoX? Is this run on 25 frames or 49 frames on EgoX? How many denoising steps where used for EgoX vs Ego-Forge?
Q1. Architecture description of R, Figures 2-3, and the positional embedding.
We thank the reviewer for the careful reading and for pointing out where our description of R was unclear.
including how h_i is fed to R) to theAppendix G.We appreciate the reviewer's constructive feedback and the opportunity to improve this part of paper. The corresponding changes are highlighted in
redin the revised PDF for ease of reference.Q2. Error propagation from the depth-reprojected prior (LL053–055).
In EgoX, depth errors affect generation through two pathways: they corrupt the ego prior and introduce errors into attention biases via GGA, compounding their effects. Ego-Forge avoids this second pathway by removing GGA, while its backbone can recover the correct viewpoint even from poor priors, as shown in our supplementary videos. We revised the abstract, highlighted in
red, to clarify this distinction.Q3. Evidence that GGA can be dropped at scale.
Please refer to our
common reply, where we address this question with the requested data-scaling ablation and provide the corresponding analysis, supporting the argument that GGA can be removed under training at scale.Q4. Is the exocentric part of the canvas noised?
We agree that this was an inconsistency in the formulation. Only the egocentric part is noised, while the exocentric part remains clean as conditioning, as shown in Figure 2. We have corrected Sec. 3.1 accordingly (highlighted in
red).Q5. Inference time in Figure 7: rendering, frame count, and denoising steps.
Frames: As already mentioned in Line 377, all timings in Figure 7 are for 25 frames, for both methods.
Denoising steps: EgoX uses 50 steps, while Ego-Forge uses 40 (as already mentioned in Line 363). We also tested 50 steps for Ego-Forge and observed no further improvement, which contributes to its speed-up.
Why rendering differs: The two pipelines use fundamentally different rendering procedures. EgoX (official repository,
EgoX-EgoPriorRenderer;scripts/render_vipe.shfor rendering) first fuses a static background point cloud across frames, then adds per-frame dynamic objects and rasterizes the merged cloud using PyTorch3D. This multi-frame fusion is the main computational bottleneck. In contrast, Ego-Forge directly renders each frame from a single MoGe-V2 point cloud, without cross-frame fusion, completing the entire prior pipeline in under 15 seconds.Please note that Figure 7 excludes EgoX's captioning time, since its exo- and ego-view captions require human or VLM input at inference, with varying associated costs. We exclude this overhead; accounting for it would further increase Ego-Forge's speed-up.
We again thank the reviewer for the thoughtful feedback, particularly the insightful questions in Q1 and Q4, which helped us further clarify and improve the paper.