acceptodds
Under review as a conference paper at ICLR 2027

TriSplat: Simulation-Ready Feed-Forward 3D Scene Reconstruction

31 views

Abstract

Sparse-view 3D reconstruction is increasingly addressed with feed-forward networks. Yet most methods use Gaussian primitives whose surfaces are only implicit. Obtaining a mesh for simulation or embodied interaction still requires costly post-hoc extraction such as TSDF fusion or Poisson reconstruction, breaking the feed-forward promise in pose-free settings where structure and pose are estimated jointly. We present TriSplat, a feed-forward network that represents scenes with oriented triangle primitives and exports meshes from a single forward pass, jointly predicting point maps, triangle attributes, camera poses, and optional intrinsics. Triangle orientation is anchored to geometry rather than regressed freely. Normals from the predicted point maps are refined by an image-conditioned head and converted into stable local frames, while a mono-normal bootstrap and opacity and blur scheduling stabilize early training and sharpen the surface. Experiments on RealEstate10K and DL3DV show more geometry-faithful reconstructions than Gaussian baselines and competitive novel-view rendering. Because the primitives are surface triangles, the output loads directly into physics engines, collision detectors, and rasterization pipelines without conversion, making it a practical simulation-ready solution for feed-forward 3D scene reconstruction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Feed-Forward 3D Reconstruction, Novel View Synthesis, Triangle Splatting

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_DMTsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Feed-forward prediction of triangle soups from up to 24 unposed input images. Input images are DINOv2 encoded, and three heads predict camera poses, triangles, and pointmaps, respectively. The architecture looks very similar to VGGT with the addition of a Primitive Head, predicting triangles.

The method is efficient, and predicts textured meshes in 1s. Compared to related work, it produces higher quality geometry and NVS on exported meshes (Table 2). With native primitive rendering, the quality is worse (Table A).

Limited novelty: Adapts VGGT to also predict triangles, and regularizes the triangles with normals derived from the predicted point cloud and a monocular normal estimator.

The approach beats similar splatting methods (in the unposed image setting), but the resolution is too coarse to be useful in games or most simulation tasks. I would tone down the simulation-ready aspects, as the method predicts a triangle soup, not watertight, manifold meshes.

Critical strengths

The method is efficient, and directly predicts textured triangle soups from unposed images in 1s.

Visual quality improves significantly in mesh-rendering comparisons against related unposed splatting work.

Critical weaknesses

Limited novelty: Adapted VGGT to also predict triangles, and regularize the triangles with normals derived from the predicted point cloud and a monocular normal estimator.

Limited resolution: One triangle per pixel at 224x224 or 224x448 if I have understood the approach correctly.

The resolution limit and the fact that the output is a triangle soup, not manifold meshes makes the actual simulation or game applications very limited.

Questions for authors

I would like to see more details about the "Local-Global Attention" block. How large is it, how many blocks? Is it trained from scratch or finetuned? The left part of the proposed architecture looks very similar to VGGT with the addition of the Primitive Head for triangle prediction, so I would recommend discussing the differences in architecture and contrast this to VGGT. For example, was the number of features or attention blocks changed to incorporate the additional Primitive Head?

Out of interest, how do the predicted pointmap and camera poses compare against VGGT?

How are the number of views varied? Is there one model trained for each of 6, 12, and 24 views, or can the same model encompass all three view counts?

For simulation and game application, the low resolution output is a severe limitation. Does the method scale higher resolutions, and what would be the limiting factors?

In Table 2, I am a bit confused why NVS quality decreases with more views (for most methods)?

In Table 2, what appearance model is used? Is this mesh rasterization with SH shading?

Notation: Several equations are repeated in the text with different notation. This should be easy to clean up.

Reviewer_dwEkOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TriSplat, a method that performs forward prediction of point clouds, camera poses, and directed triangular primitives from sparse, camera-agnostic multi-view images, and directly exports these primitives as meshes. The approach constructs local triangle orientations using the finite difference normal method on point clouds, refines these normals via an image-conditioned U-Net, and stabilizes training through early guidance from a single-view normal teacher, along with temperature and fuzzness scheduling. The authors report results on surface geometry, mesh rendering, depth/normal estimation, and inference latency across RE10K, DL3DV, and ScanNet zero-shot benchmarks, and demonstrate collision detection and interaction capabilities within Unity and Isaac Sim.

Critical strengths

The problem is well-defined and of significant importance: although feedforward Gaussian representations enable rapid rendering, they typically require additional mesh extraction before being applicable to collision detection, physics simulation, and conventional graphics pipelines. This approach employs surface triangles as primitives in both training and inference stages, ensuring direct compatibility with target output formats. The proposed solution consists of geometricly anchored orientation, monocular normal guidance, normal refinement, and progressive sharpening, which sounds effective.

Critical weaknesses
  1. The primary metric for Tables 2–3 is the rendering quality of the "exported mesh," rather than the rendering quality of the feedforward network's "3DGS" output (the actual rendering quality is provided in Appendix Tables B and D). The baseline mesh, however, was exported using the same set of TSDF parameters. For some methods, there is a significant discrepancy between the rendering quality of the exported mesh and that of the 3DGS output (as shown in Appendix Table C); does this single set of parameters truly apply to all methods? We suggest adding the following experiment: treat the original point clouds or depth maps from each baseline—without TSDF fusion—as point clouds, and evaluate them using the exact same set of metrics.
Questions for authors
  1. While ScanNet provides an official TSDF ground-truth mesh, the paper only reports depth/normal metrics on ScanNet (Table 4), omitting the mesh CD/F1 metrics. We recommend including these metrics.
  2. The task of SurfelSplat involves reconstructing an object from two images, which differs from the task in this paper. When comparing with SurfelSplat, the paper used 6/12/24 view settings, which may not be a fair comparison.
  3. In the efficiency comparison section (Section 4.4), the computation time of the feedforward network should be separated from the computation time required for subsequent mesh extraction. Other methods that do not employ grid reconstruction still execute grid extraction code after the feed-forward network, which makes them considerably slower (e.g., in the original paper of AnySplat, the 6-view feed-forward inference took 0.297s; in this paper, the combined inference and grid generation time was 18.7 s). Combining these two timings in this paper may be misleading.
  4. The paper lacks precision metrics for pose prediction (whereas AnySplat does include such metrics).
  5. Appendix J states: "all metrics are computed in the world coordinate frame." The proposed method is trained using ground-truth poses on RE10K and DL3DV; during pose-free testing, its world coordinate frame naturally aligns with that of the dataset. If the training sets of other methods do not include RE10K or DL3DV, could the coordinate frame alignment process introduce potential bias? It is recommended that the coordinate frame registration steps be elaborated in detail.
  6. The core contributions do not affect the primary geometric metrics: in Table 5, the CD for the "w/o anchoring" configuration is identical to that of the full model (both 0.190); only the Precision and PSNR metrics are affected. Further analysis is warranted to elucidate the underlying reasons.
  7. Optimization-based methods are entirely absent as upper bounds—such as Triangle Splatting (the most natural upper bound, given the origins of the primitives and rasterizers discussed in this paper) and 2DGS/RaDe-GS—leaving readers unable to assess how far the performance of feed-forward meshing methods deviates from scene-specific training results.
Reviewer_MYtbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TriSplat builds a 3D scene from a few images without knowing camera positions. It creates a set of triangles—one for every pixel—and turns them into a mesh. It uses a DINOv2 encoder and a transformer decoder to predict 3D points, camera poses, and triangle details in one step. Each triangle sits on a predicted 3D point and faces the right way using a refined normal map. The model is trained using a special "Triangle Splatting" tool. Compared to methods that use Gaussians and then turn them into meshes, TriSplat is much better at making accurate meshes and is much faster. It can make a mesh in about one second. The results are shown in Unity and Isaac Sim.

Critical strengths
  1. The design makes sense. Each triangle is placed on a 3D point that the model predicts, and it is turned to face the same way as the surface around it. This is a simple and sensible way to build a surface when the camera positions are not known. I also think it makes sense to follow the feed-forward Gaussian prediction model and update the Gaussian with faces. And the experiment clearly shows visually better than the Gaussian methods.

  2. The presentation is clear. The paper is easy to follow. Fig. 2 and Section 3 explain each part of the method with its equation, and the paper keeps native rendering and mesh rendering apart throughout and reports both, including where a baseline does better (App. C).

Critical weaknesses

W1. The "simulation-ready" claim is not tested. The title says the meshes are ready for simulation, but the paper only shows screenshots (Fig. 9, App. H) and a video. There are no numbers: no contact or collision error, and no success rate for a robot walking or navigating in the scene. The paper also does not put the Gaussian methods' meshes into the same simulator to compare. Other work does measure this: Vid2Sim (CVPR 2025) reports the success rate, a path-length score (SPL) and the number of collisions in Unity.

W2. The mesh has too many faces, which could overload a simulator. TriSplat makes one triangle per pixel, no matter how simple the scene is. Before pruning, that is 301,056 triangles for 6 RE10K photos at 224×224 and up to 2,408,448 for 24 DL3DV photos at 224×448 (my count from App. A.1; Sec. 5 says "Triangle density follows input resolution"). I don't see any discussion related to the faces in the paper, and two million faces could be a lot for simulators or renderers.

W3. Surfaces that no camera sees are missing. This would be another weakness of the pixel-based reconstruction. Each triangle comes from a pixel in one of the input photos, so anything the photos do not show has no triangles at all. The floor under a table or the back of a sofa becomes a hole, and in a simulator a ball or a robot could fall through it. As the author claimed, is this simulation-ready? Things like this could be critical for a simulation environment.

W4. The mesh is not guaranteed clean, connected surface. The triangles are not joined to each other; Sec. 5 calls the output a "non-manifold triangle soup" that is not suitable for watertight uses. When several photos see the same wall, the wall gets several overlapping layers of triangles. The whole scene is one fixed shell, so nothing in it can be moved or picked up (Sec. 1 says such objects can be added as separate assets). The scale is only relative and the up direction is unknown, so both have to be set some other way before a simulator can apply gravity correctly, and the paper does not say how this was done for Fig. 9.

W5. Some baselines could be useful for comparison. TriSplat's triangles sit on its predicted point map (Eq. 2), so it might help to compare with simple ways of turning the same point map into a mesh without TSDF. For example, neighboring pixels could be connected into triangles, as the DUSt3R demo and MoGe (CVPR 2025) do, or the points could be meshed with NKSR (CVPR 2023), which can mesh "millions of points in a few seconds". The paper chose not to include such combinations (Sec. 4.1: "we do not include these backbone–head combinations"). A comparison with MeshSplatting (CVPR 2026), which makes connected meshes that work with Unity physics, could also be informative. Such comparisons could help show how much of the gain comes from the triangles themselves and how much from skipping the TSDF step.

Overall. I think the reason behind the paper is valid, but experiments could be a little bit lacking, Including some mesh-based baselines and supporting the claim of simulation-ready. To me, this is like a 2.4 or 2.5 on a scale of 1 to 4. However, considering how many AI research papers exist in ICLR, I think human research is worth an extra 0.1 or 0.2, which makes this round up to 3.

Questions for authors
  1. Faces. How many triangles are left after pruning for different settings (6, 12, and 24 views)? How does this compare to the baseline meshes? Since the number of triangles depends on the number of pixels, how does the model handle very large scenes or high-resolution images? Can the mesh be simplified without losing quality?
  2. Occlusions. How does the model handle areas the cameras cannot see (like under a table or inside a shelf)? How does it handle areas where many cameras see the same spot? Please show results for views that are not on the same path as the input cameras.
  3. Simulation. Please provide a numerical test for simulation (like contact error or robot movement success in Isaac Sim). Please also report mesh size, collider build time, and simulation speed. How did you set the scale and gravity for the videos?
  4. Code vs. paper. I noticed three differences in the code: (i) the evaluation script uses a different scale and prunes 1% of triangles, while the paper says no information is lost; (ii) the evaluation script ignores points that are too far from the ground truth; (iii) the DL3DV ground truth used in code is a sparse COLMAP file, not a "dense reconstruction." Which settings were used for Tables 1–3, and how do the results change if you use the export method described in the paper?

Comments on the paper

4 comments
AuthorsOpenReview ↗

Response to follow-up comments on GGGS and PGSR

Thank you for suggesting GGGS and PGSR. We additionally evaluate both methods under the same setup as Q7, using 24 input views on 20 held-out DL3DV scenes. All metrics use primitive rendering. Time measures 30K steps of pure optimization for GGGS and PGSR, and one forward pass for TriSplat.

MethodPSNR ↑SSIM ↑LPIPS ↓Time (s/scene) ↓
GGGS17.070.3970.3671826.4
PGSR16.870.4080.3871707.0
TriSplat (Ours)21.130.6910.3080.7885

TriSplat achieves better results across all three rendering metrics with substantially lower computation time. We have added these results to Appendix E.5, Table M, and included references to both methods in the revised paper.

AuthorsOpenReview ↗

Response to follow-up comments on mesh reconstruction quality

Thank you for the follow-up. Primitive rendering uses the native renderer of each method, with Gaussian primitives for Gaussian-based methods and triangle primitives for triangle-based methods.

Our task targets reconstruction from sparse, unposed images. We evaluate all methods using the same 24 input images on 20 held-out DL3DV scenes, with camera poses additionally provided to the optimization-based baselines. These results characterize performance under this shared input budget. We additionally report geometric metrics for the exported meshes.

MethodCD ↓Precision ↑Recall ↑F1 ↑
3DGS0.13630.09470.24360.1363
2DGS0.16560.12430.23250.1620
RaDe-GS0.10010.06880.27510.1101
Triangle Splatting0.11290.09810.24320.1398
MeshSplatting0.21290.12420.18110.1474
GGGS0.11770.06900.27080.1100
PGSR0.21130.07440.20740.1095
TriSplat (Ours)0.09130.13580.45390.2091

TriSplat achieves the lowest CD and the highest precision, recall, and F1 under the same 24-view setting. We have added these results to Tab. M.

Thank you for the detailed response and the new experiments. Most of my concerns are addressed, and I remain positive about this paper and will keep my score of 3.

One request for the final version: could you show renderings from viewpoints that are farther away from the input views, for example side views with a large change in angle? In the current figures, some triangles near the image borders and object boundaries look very sharp, and I suspect they would look much worse when seen from the side. A comparison for viewpoints not close to the input view will make it even stronger.

AuthorsOpenReview ↗

Response to follow-up comments on side-view comparisons

Thank you for your positive assessment and helpful suggestion. We have added side-view mesh-rendering comparisons from viewpoints farther from the input cameras in Appendix K, Fig. O. The context panels explicitly mark input cameras in blue and target cameras in red. All methods are rendered at the same target viewpoints.

TriSplat better preserves wall and window structures under these viewpoint changes. These visual comparisons complement the quantitative extrapolation results in Appendix E.4, where we have also added the corresponding discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.