- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a new problem of parametric CAD reconstruction of objects, when these objects are inserted into a real environment with occlusions through walls, floors and corners. This means, that the standard paradigm of reconstructing these objects, which assumes that they are floating freely in space and that views can be selected freely does not apply anymore. The paper introduces a new, large dataset of such scenes, where CAD models from CAD-Recode and Text2CAD are inserted into SceneNet office rooms. 36 views of each object are rendered across the viewing hemisphere. The paper then proposes SlichtCAD which has a lightweight view selector, which is tasked to pick K out of 36 views, and a Qwen2-VL generator, which generates CadQuerry code. The paper has three training stages. 1. distilling a greedy coverage policy into the view selector, 2. joint training of selector and generator and 3. a generator only training. On the test split, the proposed selector beats greedy selection at every budget. The full system outperforms external CAD baselines on DeepCAD, Fusion360, T-LESS and MP6D with very low invalid-code rates.
Overall, I believe the problem is relevant, the dataset is useful, and the internal policy comparisons are carefully controlled. My main questions/concerns are: The large margins over external baselines do not isolate the contribution of view selection well enough. The “active” and “under occlusion” framing is stronger than the evidence: the gains are as large on mildly occluded floor placements as in corners. Real-capture performance depends on an author-designed canonicalization protocol and trails baselines on raw photographs.
I currently lean on borderline to weak accept, but am willing to raise my score if my concers (see Weaknesses section) are addressed
Critical strengths
- The problem is interesting, relevant and well introduced. Reconstruction of CAD objects from images alone is a challenging problem and being able to handle real world occlusions is an interesting question, which prior work largely ignores.
- I like that the paper creates a large dataset and I hope this will be released as soon as possible. Given that the dataset contains 567k objects under varying degrees of occlusion I believe this could be a valuable resource for the community. I would also encourage the authors to consider releasing the data generation protocol/code.
- The method is technically sound and the experiments are well controlled. All methods are evaluated on the identical data and results on n=4000 include paired Wilcoxon tests, bootstrap confidence intervals and a per-regime breakdown (Table 8). I also like that the policy neutral generator (Table 1b) isolates view quality from co-adaptation.
- The budget-dependent strategy (Table 5, Fig. 5) and the controlled same-scene study (Fig. 6) give a clear, interpretable picture of what the selector learns.
- The evaluation section is broad and fair. It covers synthetic CAD benchmarks and two real industrial datasets. The paper openly reports that the method trails on raw T-LESS photographs. Invalid-code rates are consistently the lowest of all methods.
- The paper is well written, the figures are informative and reproducibility is good. Architecture, warm-start targets, hyperparameters (Table 6), the alignment network and the real-capture selection protocol are described in detail, and code is provided.
Critical weaknesses
- I believe the comparisons do not fully isolate the contribution of view selection. Tables 2-4 and 9 compare full systems that differ in multiple ways (training data, generator data and number/type of input views). The +21 and +17 mIoU_eff therefore cannot be attributed to selection only. The paper’s own Table 1 suggests selection accounts for a few points. A direct way to show how much of the gap selection explains is to report the SA-max co-trained generator (with +GRPO) under the cross-method protocol on DeepCAD, Fusion360 and the 4k split.
- The active view selection and under occlusion framings are too strong given the evidence. The selector takes renders of all 36 candidate views as input, so the method assumes every feasible view has already been captured. The selector essentially does not save aquisition effort. The paper should state this explicitly, since the title and introduction suggest deciding where to look. Similarly, Table 8 shows the advantage over SA-max is at least as large for mildly occluded floor placements as for corners (e.g., +6.0 vs. +5.3 at K=4). This suggests the gain comes mostly from choosing views that make program parameters legible, not from handling occlusion. That is a valuable finding in itself, but it shifts the paper’s central message. The paper also deals with occlusion arising mainly from walls removing whole half-spaces of views rather than from partial occluders.
- The cross-method metric protocol is ambiguous. Sec. 5 says predicted and ground-truth meshes are normalized independently to the unit cube. However, the captions of Fig. 11 and later figures say the printed IoU is computed after a per-object best-fit rotation. Please clarify which protocol the tables use. A best-fit rotation would hide orientation errors and could favor methods that output arbitrary frames.
- I’m less convinced by the real-capture results, as they rely on several additional assumptions. On raw T-LESS photographs, SightCAD trails the baselines substantially (8.26 vs. 25.66 mIoU_eff for CAD-Recode←TripoSR). The favorable “canon” condition is a protocol designed by the authors themselves. The real-capture pipeline also depends on an alignment network with an exact top-1 accuracy of only 28.2%, and on camera poses taken from ground-truth object-pose annotations. I appreciate that the paper reports the raw T-LESS result openly, but I would ask the authors to reflect it in the wording of Contribution 4, which currently groups T-LESS with the benchmarks where the method leads.
- I don’t think the mechanistic conclusion is quantitatively supported. App. A.1.2 states that the supply of in-plane and face-on views “correlates strongly” with extrude-depth and sketch accuracy, but this correlation is never computed, and Figs. 7 and 9 show clear exceptions. fw-SA supplies the most face-on views (13% vs. 8% for the learned policy) but does not achieve the best sketch accuracy. Similarly, the uniform policy supplies more feasible in-plane views than SA-max (19% vs. 13%) yet has much worse extrude-depth accuracy. The paper attributes these exceptions to occluded (contaminated) views, but this explanation is not tested.
Questions for authors
- Could you report the SA-max co-trained generator (with +GRPO) under the cross-method protocol on DeepCAD, Fusion360 and the 4k split? This would show how much of the gap to the external baselines comes from view selection and how much from training on the scene-embedded data.
- Why are DeepCAD and Fusion360 evaluated on only 100 objects each? Could you report results on the full test splits, or at least confidence intervals for Tables 2–4?
- The selector takes renders of all 36 candidate views as input. Could you clarify what the selection saves in practice? As I understand it, 4. it reduces the generator’s input rather than the number of views that need to be captured.
- Table 8 shows the gain over SA-max is at least as large on floor placements as in corners. Do you see this as evidence that the benefit comes mainly from parameter legibility rather than from handling occlusion, and how does this fit the paper’s framing?
4 comments
(continued) Same generator; Δ = oblique − control, paired boot-strap 95% CI:
| Replaced view | n | Coverage (removed / control / oblique) | Δ extrude depth (pp) | Δ sketch dims (pp) | Δ IoU (pp) |
|---|---|---|---|---|---|
| In-plane (≥ 75°) | 2,212 | 0.22 / 0.24 / 0.36 | −4.5 [−6.6, −2.4] | +4.1 [+2.1, +6.1] | −0.4 [−1.5, +0.7] |
| Face-on (≤ 15°) | 623 (IoU: 719) | 0.32 / 0.31 / 0.37 | +1.6 [−1.9, +5.1] | −3.0 [−7.1, +1.0] | +0.4 [−1.1, +1.9] |
Although the oblique view covers more of the object, losing an in-plane view costs 4.5 pp of depth accuracy, so in-plane views do carry depth, which the policy-level correlation missed. Sketch information, by contrast, is not specific to face-on views: replacing one changes sketch accuracy by −3.0 pp (n.s.), and an oblique view in place of an in-plane one raises it by 4.1 pp. This explains the fw-SA exception. We will add this table to App. A.1.2, replace "correlates strongly" with these findings, and release the code (supplementary, analysis/viewswap/).
We hope this addresses the reviewer's major concerns and would be grateful if they would consider raising their score further.
Edit: The revised PDF now addresses your points: the SA-max pipeline control and complete splits with CIs (Table 2), views vs. tokens (Table 3), the scoring protocol (Sec. 5), the reworded Contribution 4 and limitations, and the measured correlations with an extended view-swap intervention (App. A.1.2, Table 7).
Update (Nov 15): the contamination explanation, tested directly.
As the reviewer noted, our explanation of the uniform exception by occluded views was untested. On the 4k test split (policy-neutral generator), we remove one feasible view from each policy's K=4 picks and fill the slot in two ways, keeping the other three views fixed:
- Feasible (control): an unused feasible view of the same view type (angle to the sketch normal: face-on ≤ 15°, oblique, in-plane ≥ 75°), with the closest visible coverage.
- Occluded: an unused occluded view (visible coverage ≤ 1%) of the same view type.
The two inputs therefore differ only in whether the object is visible.
| Policy | Objects | Extrude depth acc. (%) feasible → occluded | Sketch dims acc. (%) feasible → occluded | IoU (%) feasible → occluded |
|---|---|---|---|---|
| Learned | 1,018 | 46.2 → 41.2 (−5.0 [−8.0, −2.0]) | 40.2 → 37.9 (−2.3 [−5.3, +1.0]) | 54.1 → 51.0 (−3.1 [−4.6, −1.6]) |
| SA-max | 1,039 | 41.9 → 35.5 (−6.4 [−9.0, −3.4]) | 40.1 → 34.6 (−5.5 [−8.4, −2.6]) | 52.5 → 48.4 (−4.1 [−5.8, −2.6]) |
| fw-SA | 1,040 | 40.9 → 36.2 (−4.7 [−7.7, −1.8]) | 40.2 → 36.2 (−3.9 [−6.8, −1.0]) | 52.4 → 50.1 (−2.3 [−3.9, −0.8]) |
Paired bootstrap 95% CIs; bold: CI excludes zero. Objects: sketch planes recoverable from the GT program (1,710 of 4,000), a single extrusion axis, and both replacement types available; counts differ because the removed view comes from each policy's own picks.
An occluded view is also worse than leaving the slot empty (pooled: depth −2.5 pp [−4.0, −0.8], sketch −3.1 pp [−4.8, −1.6]), so it actively misleads the generator.
Conclusion for W5. Both exceptions are now explained by interventions rather than policy-level correlations. Uniform: an in-plane view carries extrude depth only if the object is visible in it, so uniform's frequent occluded picks (Fig. 9) cancel its larger supply of in-plane views. fw-SA: its extra face-on views add little because oblique views already carry much of the sketch (Table 7). App. A.1.2 replaces "correlates strongly" with these interventional results, and we will add this experiment to it.
Response
I would like to thank the authors for the unusually thorough and candid rebuttal. The SA-max cotrained pipeline controll of the full splits addresses my main concern. The clarifications on the scoring protocolm the view swap and the revised contribution statement mainly address my leftover concerns. Overall I think it is good that the work is getting reframed around budgeted view selection.
A few remaining points:
- Since the gain does not come mainly from handling occlusion, please also revise the title and abstract (“Active View Selection … Under Scene Occlusion”, “view feasibility as a first-class constraint”) so they match the new framing
- On the industrial sets, the rebuttal reports the selection gain against the SA-max pipeline (+5.7 on canonicalized T-LESS, +2.9 on MP6D real). With the generator held fixed, Table R shows only +1.8 and +0.4, and on synthetic T-LESS, SA-max views do better (56.5 vs. 54.0). Please report the fixed-generator numbers and state that selection gives no clear benefit on the real-image sets.
One more question:
- Feasibility is only learned, and occluded views actively mislead the generator. How often does the learned selector pick an occluded view at test time?
The rebuttal did a great job overall at clarifying the scope of the paper and I am happy to raise my score to reflect this.
We sincerely thank the reviewer for the score increase and for the kind words. The reviewer's questions led to the controls, the view-swap interventions and the reframing that made the paper substantially stronger, and we are grateful for the careful engagement throughout.
Title and abstract. We agree. The title will become "Look Where You Can: Budgeted View Selection for CAD Reconstruction of Scene-Embedded Objects", and the abstract will replace "treats view feasibility as a first-class constraint" with a description of the budgeted setting: the scene determines which views are available, and the selector chooses the K that make the program's parameters legible to the generator.
Fixed-generator results on the industrial sets. Tables 4–5 will now report the same-generator comparison (learned vs. SA-max views, Table R) alongside the pipeline comparison, and Sec. 5 and the limitations will state that, with the generator held fixed, learned selection shows little benefit on the industrial sets.
How often does the learned selector pick an occluded view? Rarely. On the 4k test split at K=4, 0.6% of learned selections contain an occluded view (0.2% of all picked views): 0.8% for floor, 0.2% for wall and 0.6% for corner placements. We will report this in App. A.1.2.
Response to Reviewer xpte
We thank the reviewer for the constructive review. We ran the requested control, evaluated every method on the complete DeepCAD and Fusion360 splits (Table R, general response), and computed the missing correlation. The updated supplementary adds the code for the new experiments, and a revised PDF of the main paper incorporating all changes described below will be uploaded shortly.
W1/Q1: How much of the external gap comes from view selection?
We scored the SA-max co-trained generator (+GRPO) with its own views under the cross-method protocol (paired 95% CIs):
As suspected, in-domain training explains most of the external gap: the SA-max pipeline alone leads cadrille by +16 to +23. Learned selection adds a significant +2.8 to +5.3 on top, on the scale of Table 1, and +5.7 / +2.9 on canonicalized T-LESS and MP6D real. The paper will report this split explicitly.
Q2: Can you report the complete test splits, with confidence intervals?
The 100 + 100 objects were a fixed random draw, sized by the slowest baselines. Table R now evaluates every method on the complete splits with paired CIs, except CAD-Fit (~5 min per object). The ranking is unchanged: our margin over the strongest baseline is +26.4 / +18.9 with native cameras, and +21.3 / +16.2 when blocked cameras are replaced by feasible views. Tables 2–4 will report both.
W2/Q3: What does view selection save in practice?
We agree that K limits what the generator reads, not what is captured, and will rename the setting budgeted view selection. Selection pays off at inference (Table K): all ~24 feasible views score 22.5 against 60.5 at K=4, which uses 5.9× fewer prompt tokens. The forward pass takes 43 ms at K=4 against 198 ms with all feasible views (batch size 1, one H200); the selector adds 7 ms.
W2/Q4: Does the gain come from legibility rather than from handling occlusion?
Yes, and we will adopt this as the central message. Since SA-max is itself occlusion-aware, the gain comes from the legibility of the chosen views, and it is significant in every regime (Table 8). Occlusion instead defines the candidate set and breaks fixed camera rigs. We will note that our occlusion is dominated by walls and floors, and list partial occluders as a limitation.
W3: Which protocol do the cross-method tables use?
We do not rotate per object when computing metrics. Each mesh is normalized to the unit cube, and one axis rotation per method is fixed for all its objects to absorb conventions such as y-up versus z-up, which penalizes rather than favors arbitrary per-object frames. Table 1 compares our own programs with exact mesh IoU and no rotation; the cross-method tables use 32³ voxel IoU because baseline meshes are often not watertight, and re-scoring them with exact IoU only widens our margins. The best-fit rotation only aligns the solids in the qualitative figures. We will state this; the scorer is in the supplementary.
W4: How reliable are the real-capture results?
The reviewer is right to question it and we will reword Contribution 4: SightCAD leads on DeepCAD, Fusion360, synthetic T-LESS and MP6D and canonicalized captures, but trails on raw T-LESS photographs. Canonicalization will be presented as our diagnostic of the image-domain gap, and we will list every ground-truth input used; the selector and generator never see the GT mesh. Indeed our Align-net's low exact top-1 (28.2%) reflects part symmetry: it is correct up to a 90° rotation 90.3% of the time, and residual errors count against us. We would like to mention though that canonicalization is still useful to our method in case of actual deployment.
W5: Is the mechanistic conclusion quantitatively supported?
Over 30 cells (six policies × K=1–5, same policy-neutral generator), face-on supply predicts sketch accuracy (r = 0.67, p < 1e-4) but in-plane supply does not predict depth (r = 0.19, n.s.); each occluded view costs 3.7 pp depth and 2.9 pp sketch. Because policy-level correlations confound view type with coverage, we added an intervention on all 8,443 held-out objects. In the learned K=4 picks, one view is replaced, with the other three fixed, either by a feasible view of the same type (control) or by a feasible oblique view (15–75° to the nearest sketch normal), each the closest match in visible coverage.