- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose using network-provided update directions for radiance foam optimization (with respect to sparse conditioning views) rather than direct gradient descent on the parameters. They show that these are much more efficient (~10s to beat ~60 minutes of direct optimization). Among others, they discuss creating artificial reference images (through gaussian splatting reconstruction), as well as interleaved appearance-geometry optimization.
Critical strengths
- the approach makes intuitive sense to me: the gradients from radiance foam differentiable rendering are too chaotic for efficient gradient descent optimization, but scattering local error information through the foam allows a NN to learn effective update steps anyway.
- the authors show improvements both on 8-view and 2-view sparse reconstruction, as well as generalization capability across datasets.
- the most crucial ablation studies are present
Critical weaknesses
- none of the improvements are motivated are discussed; they are all presented as "we randomly tried this and luckily it worked better than the baseline". Ditto with the baseline, and much of the math exposition: only someone already familiar with the PowerFoam optimization approach would be able to follow it.
- existing work discussion is too succinct and impossible to follow for people not intimately familiar and its history with the field already. Radiance foam solutions are not even discussed in the first two paragraphs, nor is a clear discussion of its benefits/drawbacks given at any point.
- the selection of 7 midpoint cameras for the pseudo imagery + 14 more (virtual) intermediate centerpoints (L226 and onwards) is not motivated at all and feels like optimizing for line-based sparse view captures (whose datasets often only evaluate on the full trajectory). A more holistic approach that would consider e.g. circular trajectories (where pseudo views within that circle would be more informative) is more appropriate.
- the last section about rendering flexibility of radiance foam, while correct, is implicitly misrepresented as a benefit of this work, when it applies to all radiance foam / power foam methods.
Questions for authors
-
I have significant concerns about the use of AI for writing this paper: in many spots, counter-intuitive wording is used, or field-established terminology is actively avoided. I also spotted several instances of "AI filler" sentences, which add nothing to the local text except confusing the reader. They are too cryptic or actively irrelevant to that section of the paper --> independent references (L076, later as well): implies these are reference images, which they are NOT. These are artificially/heuristically created interpolated view positions, i.e. "pseudo" views. --> L127: "The distinction matters ..." that is true, but why does "a structured subproblem" give us any benefit whatsoever? --> L143: "The panels are analytical schematics." a pointless sentence. --> L150: "An oriented surface separates a side of density sigma from an empty side." Again true, but not revisited anywhere in the manuscript, nor is it relevant to the current paragraph. --> L247: the distinction between "image residuals" and "gradients of the source-image loss" is nonexistent and there is no point in trying to make one --> L284: "development scenes" = validation scenes, "confirmation scenes" = testing scenes
-
General (smaller) manuscript comments, which unfortunately do add up: --> L080: Transfer(s) --> L082: mention the (very positive!) comparison against ~1 hour optimization for PF. --> L118: referring to SceneFoam (the current manuscript) as "their" is a typographical error --> Fig2: unclear how the notations here map onto (g, a, C). Should only occur after the first mention in the text. --> In general, the way to select directions for directional color bases is too opaque and not reproducible from this manuscript on its own. --> How to select the texture site locations (L151) is not explained at all. How to map a sphere-topology onto a material patch is not obvious at all --> L158: where does the offset come from? How would any aspect of the appearance be independent of the color selection? This implies e.g. we could never make the scene fully dark? Eqn2 is further missing any directional dependency, when further on the manuscript discusses that each direction has a color attached. --> L202: I found the discussion around finetuning ReSplat for a now output modality confusing; I suggest formulating it as 'reinterpreting the ReSplat output as radiant foam parameters and finetuning from there' rather than talking about intermediate gaussian outputs --> L205: Unclear how P_seed fits into Fig 3. --> L208-209 was a bit too unclear, given that it reads like it applies only to the colouring optimization (which per Eqn2 and Sec4.2 is a "structured subproblem" which is easily solved) --> L221: observed colors instead of predicted colors? --> L241: is the decision whether or not to take an update step a local decision (cell-per-cell) or global? --> L250: why normalize the gradients? There is no motivation given, nor is this ablated --> L311: it is not clear to me whether RealEstate has a dedicated model trained for it, nor which model exactly is used for ACID. --> Sec 5.3: not clear from reading (only from number comparisons) on which datasets these ablation studies happened --> Sec 5.3 "color fitting before refinement": I'd also have much preferred to see the post-refinement impact of omitting the color fitting before refinement
2 comments
Thank you for these additional details. I will be retaining my positive score.
Dear Reviewer UkE5,
Thank you for your suggestions on camera selection and presentation. Following your feedback, we tested a coverage-based camera strategy, which improved PSNR by 0.22 dB on the eight partial-orbit scenes (Table U2b). We also outlined how we’ll clarify the Power Foam background and pseudo-view terminology. If you have a chance to look at our response, we’d welcome any feedback on these points.