- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper explores a unified flow-based approach that allows a single model to perform reconstruction with one inference step and conditional generation through multiple steps. The authors investigate this idea using toy examples and two 3D tasks, namely novel view synthesis and geometry prediction. The experiments suggest that multi-step inference can improve perceptual image quality and certain geometric boundary metrics, although these gains sometimes come at the cost of pointwise prediction accuracy.
Critical strengths
1. The paper addresses a relevant problem.
Understanding how reconstruction and generation can complement each other is an interesting direction, particularly given the practical tension between making accurate predictions and producing visually plausible, scene-consistent outputs.
2. The evaluation spans different tasks and output spaces.
The toy experiments help illustrate the proposed mechanism in a controlled setting, while the novel view synthesis and geometry experiments provide a broader picture of how the approach behaves on realistic 3D prediction tasks.
3. The evaluation goes beyond standard reconstruction accuracy.
In addition to PSNR, the authors report perceptual metrics such as LPIPS and FID, as well as measures of geometric boundary quality and artifacts. This makes it easier to understand where iterative inference helps and what is potentially lost in terms of reconstruction accuracy.
Critical weaknesses
1. Limited novelty of the theoretical contribution.
The paper builds on the observation that target-independent noise provides no additional information about the target once the context is given. As a result, the optimal one-step predictor is the same as in standard regression, reducing to the conditional mean under squared error. With multiple steps, state-dependent predictions can instead produce conditional samples. These observations are fairly natural from the perspective of statistical decision theory and conditional flow models. Although the paper cites relevant prior work, it remains unclear what the new technical insight is beyond existing research on conditional generation, diffusion-based reconstruction, and the perception–distortion tradeoff. Applying these principles to two 3D tasks is interesting, but does not by itself seem sufficient to support the broader claims of a unified framework made in the title and abstract.
2. The theoretical guarantees do not fully carry over to the actual models.
The idealized flow transport result in the appendix relies on an optimal conditional mean field under squared error, suitable ODE assumptions, and exact integration. In practice, both the image and geometry experiments use finite-capacity networks, a limited number of sampling steps, and additional training losses. The authors also acknowledge that, under general task-specific losses, the optimal clean-target prediction need not coincide with the optimal squared-error flow field. The geometry sampler further introduces a normalization projection. Therefore, while the theory offers a plausible explanation for why iterative generation might help, it does not establish that the improvements of JUSt3R and JiT-LVSM arise from the same underlying mechanism.
3. The image experiments do not isolate the effect of iterative sampling.
LVSM is trained as a one-step predictor, whereas JiT-LVSM uses flow matching together with additional image-space losses. The appendix also describes configurations involving pixel MSE and perceptual losses. Thus, despite sharing the same backbone, the two models differ in more than just the number of inference steps. The results are also somewhat mixed. JiT-LVSM has roughly 1.7–1.8 dB lower PSNR under sparse context, while its PSNR advantage under richer context is only about 0.6 dB. Improvements in LPIPS and FID are encouraging, but they do not necessarily imply better scene fidelity. A sharper or more realistic-looking image may still contain plausible details that are inconsistent with the actual scene, as the authors themselves acknowledge in the sparse-context setting. A more controlled comparison is needed to determine how much of the improvement comes specifically from iterative sampling rather than the changed training objective.
4. The geometry experiments have a similar attribution issue.
The paper presents the one-step endpoint of JUSt3R as a counterpart to DUSt3R's direct prediction, but their training objectives are not exactly matched. As noted in the appendix, endpoint equivalence requires matching the full loss, whereas the complete flow training objective is not simply equivalent to DUSt3R's geometry regression loss. The authors acknowledge this limitation, but its implications are not sufficiently addressed in the experimental analysis. Consequently, comparable or better one-step performance does not necessarily demonstrate that flow training preserves the reconstruction ability of direct regression. Likewise, the multi-step gains cannot be cleanly attributed to iterative generation without separating the effects of the training objective and the sampling procedure.
Questions for authors
See weaknesses.