acceptodds
Under review as a conference paper at ICLR 2027

Density-Guided Variational Semantic Field Recovery for Open-Vocabulary 3D Part Segmentation

Abstract

Point–text matching based methods for open-vocabulary 3D part segmentation di- rectly convert prompt-conditioned responses into part predictions, making them vulnerable to low-density semantic outliers and boundary-mixed responses in the underlying semantic field. To address this issue, we formulate the task as prompt- conditioned semantic field recovery before final prediction and propose DGV- SFR. First, density-guided semantic denoising moves unreliable point states to- ward local prompt-consistent high-density modes while suppressing propagation across risky part boundaries. Second, a unified variational Dirichlet–Wiener cor- rection jointly models the nonparametric posterior semantic surface and point- wise uncertainty, and evolves uncertain states on this surface under boundary and anchor constraints. The recovered semantic field is finally used for standard masked point–text matching, preserving the original open-vocabulary prediction interface. Experiments on standard benchmarks show that DGV-SFR achieves state-of-the-art average performance, improving a strong point–text matching baseline by approximately 4.1 mIoU. Further ablations, calibration analysis, and boundary-missing stress tests show that the recovered field reduces low-density er- rors, improves uncertainty calibration, and better preserves segmentation quality when local boundary evidence is incomplete.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Open-vocabulary 3D part segmentation,Probabilistic modeling,Variational inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_77EZOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the errors in open-vocabulary 3D part segmentation that arise when noisy point features and boundary-mixed responses near part boundaries are directly passed to the point-text matching stage. The paper proposes a method named DGV-SFR, which first performs prompt-conditioned, boundary-aware local denoising, and then leverages a truncated Dirichlet process mixture model and score-driven Wiener dynamics to correct uncertain point states. Final prediction employs the original masked point-text matching rule.

Critical strengths
  1. The paper targets specific failure modes and provides controlled evidence of performance gains.
  2. The evaluation covers diverse and useful test conditions while preserving a practical prediction interface.
Critical weaknesses
  1. The described inference dynamics fail to substantiate a core part of the conservative correction interpretation. The drift contains an anchor term proportional to Yi(t)−yi0Y_i(t)-y_i^0, yet each block starts from Yi(0)=yi0Y_i(0)=y_i^0 and uses only one deterministic inference step (Eq. (6); l270-272; Algorithm A2). Consequently, at this step, the anchor term is identically zero. The lower mIoU of the separately trained "w/o anchor tethering" variant (Table 5) does not prove that anchoring directly constrains inference-time motion. Furthermore, Eq. (9) only proves local posterior-density ascent for the continuous-time, score-only trajectory; it does not prove that the full finite-step update preserves part boundaries. This is critical because conservative, boundary-preserving motion is claimed as the primary reason for the method's performance gains.

  2. The probabilistic coupling between the mixture posterior and the correction path is not sufficiently specified to justify the claimed unified variational mechanism. The path drift in the generative factor uses a density constructed from the current variational posterior (Eqs. (5)–(7)), while the approximate posterior also includes a path factor. Appendix C.4 appears to describe both path factors as using the same Euler Gaussian transitions. If the variational path factor uses the same transition as the generative factor, the path KL may vanish or become degenerate; if they differ, the respective distributions and the path KL divergence need to be explicitly specified. Moreover, the point-wise uncertainty uiu_i is computed deterministically from variational statistics rather than being represented as an independent factor in Eq. (7). The ELBO ablation shows that the overall objective matters, but fails to disentangle which part of the claimed joint trajectory inference actually drives the performance improvement. This ambiguity affects my assessment of the paper's primary theoretical contribution.

Questions for authors
  1. What are the exact number of training steps TT and step size Δt\Delta t? Does inference indeed execute exactly one Euler step per recovery block as described? On a fixed trained checkpoint, please compare inference results with the anchor term enabled versus disabled, and report the mean mIoU and boundary metrics. If the claimed inference benefit requires more than one step, a small-scale controlled sweep over the number of deterministic inference steps TinfT_{\mathrm{inf}} would help clarify the discrepancy with Table 5.

  2. Please present the generative and variational path transition densities side by side, clarify whether the generative transition depends on the current variational parameters, and explicitly show the path contribution to the ELBO. Is this contribution non-zero during training? If so, please report an ablation study that removes only the path-specific term while retaining the mixture ELBO, segmentation loss Lseg\mathcal{L}_{\mathrm{seg}}, and boundary supervision Lbnd\mathcal{L}_{\mathrm{bnd}}.

  3. For the described one-step inference update, using the existing test protocol, please report the fraction of corrected points for which the modeled log-density decreases, along with the boundary leakage rate and boundary mIoU. Comparing a fixed checkpoint with and without the boundary projection operator PilP_i^l would help disentangle the operator's direct effect on inference-time motion from the effect of retraining the model after removing this term in Table 5.

5 comments

Dear Reviewer 77EZ,

Thank you for examining the inference dynamics and probabilistic formulation. We answer each weakness and question separately below.

[W1] Anchor displacement and the scope of boundary claims

Each block starts at Yiℓ,0=yiℓ,0Y_i^{\ell,0}=y_i^{\ell,0} and uses one deterministic inference step, so the anchor force −λ(1−uiℓ)(Yiℓ−yiℓ,0)-\lambda(1-u_i^{\ell})(Y_i^{\ell}-y_i^{\ell,0}) is zero at that step (Sec. 3.3, Eq. (6); Appendix C.3). The separately trained Table 6 variants (36.82 and 36.22 mIoU) therefore do not demonstrate a direct fixed-checkpoint inference effect. Equation (9) proves local density ascent only for the continuous-time score component; it does not guarantee boundary preservation by the full discrete update (Appendix C.5). The projection diagnostics support only an empirical reduction in boundary leakage under the evaluated protocol.

[W2] Mixture–path coupling and pointwise uncertainty

The variational mixture defines the posterior semantic surface, whose score drives the Wiener correction (Sec. 3.3, Eqs. (4)–(9); Appendix C.2–C.4). Pointwise uncertainty is computed from assignment entropy and posterior covariance, not represented by an independent variational factor. Because the submitted generative and variational path kernels coincide, their explicit log-ratio is zero. The correction ablations (Sec. 4.4, Tables 4–6) support the complete module but do not isolate a path-only effect; we therefore make no claim of a separate Kullback–Leibler contribution for the path.

[Q1] Training/inference steps, fixed-checkpoint anchor comparison, and step sweep

Training uses Ttrain=2T_{\mathrm{train}}=2 Euler steps with Δt=0.5\Delta t=0.5; the training noise scale is νtrain=0.006\nu_{\mathrm{train}}=0.006. Default inference uses one deterministic Euler update per recovery block, Tinf=1T_{\mathrm{inf}}=1, with νinf=0\nu_{\mathrm{inf}}=0 (Sec. 4.1; Appendix E.3). Under the default one-step update, fixed-checkpoint outputs with and without the anchor are mathematically identical because the anchor force is zero at initialization. Their metrics therefore equal the default-run values: overall mIoU 36.824, boundary mIoU 32.800, leakage 11.500%, and density-decrease rate 6.150%. This equality follows analytically from Eq. (6), rather than from an independently measured anchor ablation.

The sweep varies only the number of deterministic inference updates per block:

TinfT_{\mathrm{inf}}CoarseFineShapeNetPartPartNetEOverallLatency (ms/sample)
155.84032.29537.46021.70336.8246.32
255.70032.18037.35021.68036.7287.04
455.25031.96037.10021.55036.4658.41

For one, two, and four steps, displacement/boundary mIoU/leakage/density gain are 0.061/32.8/11.5%/+0.460.061/32.8/11.5\%/+0.46, 0.104/32.5/12.0%/+0.520.104/32.5/12.0\%/+0.52, and 0.171/31.9/13.4%/+0.570.171/31.9/13.4\%/+0.57. More steps increase movement and leakage while reducing boundary mIoU; one step is a conservative default, not a unique optimum. We have reported these diagnostics in Appendix F.3.

[Q2] Generative and variational path transitions and the ELBO contribution

Conditioned on the same current variational surface and drift, the generative and variational transitions are the same Euler--Gaussian kernel (Sec. 3.3, Eqs. (5)--(8); Appendix C.3--C.4):

p(Yil,t+1∣Yil,t,.)=qphi(Yil,t+1∣Yil,t,.)=N(Yil,t+Deltatfphil(Yil,t,uil,Pil),(nuil)2DeltatIh)p(Y_i^{l,t+1}|Y_i^{l,t},.)=q_phi(Y_i^{l,t+1}|Y_i^{l,t},.)=N(Y_i^{l,t}+Delta t f_phi^l(Y_i^{l,t},u_i^l,P_i^l),(nu_i^l)^2 Delta t I_h)

The drift f_phi^l depends on the current variational mixture through its posterior surface. The path contribution to the evidence lower bound is zero because these conditional kernels coincide. Let q denote q_phi^l. The four remaining ELBO terms are:

Lstickl=Eq[logpalpha(Vl)−logq(Vl)]L_{stick}^l=E_q[log p_alpha(V^l)-log q(V^l)]

Lcomponentl=Eq[logp0(θl)−logq(θl)]L_{component}^l=E_q[log p_0(\theta^l)-log q(\theta^l)]

Lassignmentl=Eq[logp(Cl∣Vl)−logq(Cl)]L_{assignment}^l=E_q[log p(C^l|V^l)-log q(C^l)]

Lobservationl=Eq[logp(Yl∣Cl,θl)]L_{observation}^l=E_q[log p(Y^l|C^l,\theta^l)]

Here V^l denotes the stick-breaking variables, \theta^l the component parameters, C^l the categorical component assignments, and Y^l the observed initial semantic states. Thus, LsticklL_{stick}^l, LcomponentlL_{component}^l, and LassignmentlL_{assignment}^l respectively account for the stick, component, and assignment prior--posterior contributions, while LobservationlL_{observation}^l is the likelihood of the observed states. The resulting decomposition is:

LELBOl=Lstickl+Lcomponentl+Lassignmentl+Lobservationl+0L_{ELBO}^l=L_{stick}^l+L_{component}^l+L_{assignment}^l+L_{observation}^l+0

The final 0 is the Wiener-path contribution, which vanishes because the generative and variational path kernels coincide. The component prior is defined by:

p0(Θℓ)=∏mG0(Θmℓ)p_{0}(\Theta^{\ell})=\prod_{m}G_{0}(\Theta_{m}^{\ell})

and is independent of the stick variables. The path term is zero because the conditional kernels coincide; segmentation and boundary log-likelihoods are added outside the ELBO (Sec. 3.4, Eqs. (10)--(11); Appendix C.4). Thus there is no nonzero path-only term to ablate. We have clarified the factorization and described the contribution as posterior-surface estimation with constrained Wiener correction.

[Q3] Density decrease, boundary leakage, boundary mIoU, and projection

In the fixed-checkpoint comparison, a point is corrected when its state displacement satisfies:

∥YiT−Yi0∥2>10−3\|Y_i^T-Y_i^0\|_2 > 10^{-3}

The boundary band contains points with at least one different-part neighbor among 24 neighbors; the neighborhood size is:

kb=24k_b=24

Leakage is the fraction of boundary-band updates directed toward an incompatible part. Density decrease is the fraction of corrected points satisfying:

log⁡ρϕ(YiT)<log⁡ρϕ(Yi0)\log\rho_{\phi}(Y_i^T) < \log\rho_{\phi}(Y_i^0)

mIoU averages four prompt/pose settings; boundary metrics use canonical Part-of-Object prompts.

BenchmarkProjectionMean mIoUBoundary mIoULeakage (%)Density decrease (%)
CoarseOff55.80039.19.83.8
CoarseOn55.84040.18.23.7
FineOff32.20024.313.55.2
FineOn32.29525.011.75.1
ShapeNetPartOff37.35032.714.47.1
ShapeNetPartOn37.46033.812.17.0
PartNetEOff21.62031.115.59.0
PartNetEOn21.70332.314.08.8
Macro averageOff36.74331.80013.3006.275
Macro averageOn36.82432.80011.5006.150

Projection changes overall mIoU slightly, improves boundary mIoU, and lowers leakage on all four benchmarks. This supports an empirical reduction in risky boundary motion, not a universal boundary-preservation guarantee (Appendix C.5; Appendix F.3).

I have reviewed the rebuttal and additional diagnostics, which resolve my initial technical questions regarding the one-step inference dynamics and step sweep, the vanishing ELBO path term, and the fixed-checkpoint boundary projection and density-decrease evaluations. However, these clarifications—including the zero path KL contribution, the inactive anchor force during default inference, and the newly disclosed supervised training on 3DCoMPaT200 (which is not supervision-matched with Table 1's zero-shot baselines)—substantially narrow the paper's theoretical and empirical claims, while a simpler boundary-aware refinement baseline under matched supervision is still needed. Because properly integrating these major revisions and protocol alignments requires a thorough reorganization beyond the rebuttal period, I maintain my rating of 2 (Weak rejection) and encourage the authors to revise and resubmit.

Reviewer_HLQCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DGV-SFR for open-vocabulary 3D part segmentation. Before standard masked point–text matching, point features are refined through boundary-aware, density-guided denoising and a variational Dirichlet–Wiener correction. The latter moves uncertain states along the score of a truncated-DP mixture density, with boundary projection and anchoring. The paper reports a 4.10-point mIoU improvement over an internal direct-matching baseline (32.72 → 36.82), alongside results on four benchmarks, ablations, calibration analysis, and boundary-missing evaluations.

Critical strengths

• Refining point features before text matching is a reasonable approach to isolated semantic errors and ambiguous part boundaries. • The reported improvements cover four benchmarks and different prompt and pose settings. The paper also examines calibration and boundary behavior. • The theoretical claims are appropriately qualified. Eq. 9 covers only the score component, and the limitations section acknowledges failures on thin or weakly separated parts.

Critical weaknesses
  1. The role of the joint variational correction is unclear. Appendix C.4 describes the variational path factor as the product of the Gaussian transitions in Eq. 26, which also define the generative path likelihood. If these conditional distributions are identical, the corresponding path terms cancel in the ELBO. The correction can still receive gradients through downstream objectives, but it is unclear what additional variational constraint is imposed on the path itself. The authors should distinguish the two distributions and show the expanded objective. The anchor term raises a separate concern. Default inference uses one Euler step per block, initialized at (Y_i^{\ell,0}=y_i^{\ell,0}). The anchor displacement is therefore zero at the only update, so anchoring has no direct effect during default inference. Table 5 nevertheless reports a 0.60-point improvement from anchoring. This could result from multi-step training, but the training step count and whether the ablation involves retraining are not specified.
  2. Essential training details are missing. The paper does not clearly identify the initial feature extractor and checkpoint, the text encoder, or the data used to train the recovery modules. It is also unclear whether one model is evaluated across all benchmarks or separate models are trained for individual datasets. Eqs. 10–11 require point-level semantic labels and boundary supervision, so the source of these labels and their relationship to the evaluation sets are necessary for assessing open-vocabulary generalization and comparisons with prior work. The internal shared-feature comparison is useful, but shared features alone do not establish matched supervision. The paper needs to specify the training, validation, and test splits, the frozen and trainable components, and the main training settings.
  3. Some experimental results require reconciliation, and important controls are insufficiently specified. In the boundary-missing experiment, several baselines improve sharply over the corresponding clean-input results. Under the stated canonical Part-of-Object setting, PointCLIPV2 on 3DCoMPaT200-Coarse increases from 14.09 in Table 1 to 31.54 in Table 2, while PartSLIP++ on ShapeNetPart increases from 1.43 to 16.91. Removing points need not always reduce accuracy, but changes of this magnitude require an explanation. A zero-deletion evaluation using the same checkpoints, preprocessing, test samples, and evaluator as Table 2 would establish whether the two tables are comparable. The finite-mixture baseline is also insufficiently specified, and there is no clear comparison with the same correction architecture trained deterministically. A matched finite-mixture baseline and a deterministic-training variant would help establish whether the DP prior and Wiener noise are needed beyond the refinement architecture itself.
Questions for authors

Please refer to weakness

4 comments

Dear Reviewer HLQC,

Thank you for identifying issues in the variational objective, inference dynamics, training protocol, and evaluation controls. We address each weakness and question individually below.

[W1] Joint variational correction, path term, and anchor

In the submitted factorization, the generative and variational paths use the same conditional Gaussian Euler kernel (Sec. 3.3, Eqs. (5)--(8); Appendix C.3--C.4), so the path contribution to the evidence lower bound (ELBO) cancels:

p(Yil,t+1∣Yil,t,.)=qphi(Yil,t+1∣Yil,t,.)=N(Yil,t+Deltatfphil(Yil,t,uil,Pil),(nuil)2DeltatIh)p(Y_i^{l,t+1}|Y_i^{l,t},.)=q_phi(Y_i^{l,t+1}|Y_i^{l,t},.)=N(Y_i^{l,t}+Delta t f_phi^l(Y_i^{l,t},u_i^l,P_i^l),(nu_i^l)^2 Delta t I_h)

Here q denotes q_phi^l. The four named ELBO terms are defined as follows:

Lstickl=Eq[logpalpha(Vl)−logq(Vl)]L_{stick}^l=E_q[log p_alpha(V^l)-log q(V^l)]

Lcomponentl=Eq[logp0(θl)−logq(θl)]L_{component}^l=E_q[log p_0(\theta^l)-log q(\theta^l)]

Lassignmentl=Eq[logp(Cl∣Vl)−logq(Cl)]L_{assignment}^l=E_q[log p(C^l|V^l)-log q(C^l)]

Lobservationl=Eq[logp(Yl∣Cl,θl)]L_{observation}^l=E_q[log p(Y^l|C^l,\theta^l)]

Here V^l contains the stick-breaking variables, \theta^l the component parameters, C^l the categorical component assignments, and Y^l the observed initial semantic states. Thus, LsticklL_{stick}^l is the stick prior--posterior contribution, LcomponentlL_{component}^l is the component prior--posterior contribution, LassignmentlL_{assignment}^l is the assignment contribution, and LobservationlL_{observation}^l is the observed-state likelihood contribution. The Wiener-path contribution is the expected log-ratio of the path kernels and equals zero because the generative and variational path kernels coincide. Therefore,

LELBOl=Lstickl+Lcomponentl+Lassignmentl+Lobservationl+0L_{ELBO}^l=L_{stick}^l+L_{component}^l+L_{assignment}^l+L_{observation}^l+0

The mixture determines the posterior surface and Wiener drift, and the recovered terminal state enters the supervised objective; the path adds no variational constraint. We have clarified this limitation and do not claim a separate Kullback–Leibler contribution for the path. Segmentation and boundary log-likelihoods remain outside the evidence lower bound (Sec. 3.4, Eqs. (10)–(11); Appendix C.4).

Training uses Ttrain=2T_{\mathrm{train}}=2, Δt=0.5\Delta t=0.5, and νtrain=0.006\nu_{\mathrm{train}}=0.006. At inference, each block starts at Yiℓ,0=yiℓ,0Y_i^{\ell,0}=y_i^{\ell,0} and takes one deterministic step, so the anchor force is zero at that step. Table 6 compares separately trained variants (36.82 and 36.22 mIoU), not fixed-checkpoint inference with the anchor toggled. Equation (9) proves local ascent only for the continuous-time score-only component, not boundary preservation by the complete finite-step update (Appendix C.5).

[W2] Training setup, supervision, and evaluation protocol

The recovery modules are trained on 3DCompat200 with point-level labels for segmentation and neighboring-label differences for boundary supervision. The point feature extractor follows the CoSMo3D Point Transformer protocol and is frozen, as is the SigLIP Base Patch16-224 text encoder; recovery blocks, boundary predictor, projection head, and variational surface parameters are trainable. Training and validation manifests contain 27,280/1,420 objects before filtering and 26,759/1,400 usable objects afterward. Evaluation uses 3DCompat200-Coarse/Fine, ShapeNetPart, and PartNetE (Sec. 4.1; Appendix E.3–E.4). No evaluation-object basename overlaps the recovery-training manifest, while exact category/part-name overlap is 200/200 and 988/989 for Coarse/Fine, 9/16 and 13/41 for ShapeNetPart, and 26/45 and 28/40 for PartNetE.

The same DGV-SFR recovery checkpoint is used for all four benchmarks; no benchmark-specific recovery model is trained. We have made this shared-checkpoint protocol explicit in Sec. 4.1. The internal direct-matching baseline controls for adding recovery under shared features and evaluation; literature baselines retain their reported protocols and do not constitute matched-supervision comparisons (Sec. 4.1, Table 1).

[W3] Boundary-missing comparisons and mechanism controls

Table 1 reports four prompt/pose settings, whereas Table 3 reports only the canonical Part-of-Object setting (Secs. 4.1 and 4.3). The DGV-SFR 0% row is the no-removal result for that setting: it matches the Ours row’s Canonical Part-of-Object cells in Table 1 (52.36, 29.18, 38.08, and 20.74 for Coarse, Fine, ShapeNetPart, and PartNetE). Its four-benchmark mean is 35.09; Table 1’s overall mean of 36.824 averages all four prompt/pose settings. DGV-SFR uses the same recovery checkpoint across all four benchmarks; only the evaluation point cloud is altered to create the 8% condition, with no retraining or additional input processing. CoSMo3D and DGV-SFR form the protocol-aligned cross-method comparison. PointCLIPV2, PartSLIP++, and Find3D are adapted stress-test reference runs. Their 0% entries are paired clean controls from those adapted runs, not the corresponding literature values in Table 1; because preprocessing and evaluator alignment differ, they are included for context and are not protocol-matched evidence.

Accordingly, the Table 3 DGV-SFR 0% row is the same experiment as Table 1’s Canonical Part-of-Object cells for Ours; the difference between the means (35.09 versus 36.824) comes only from averaging four cells versus all sixteen prompt/pose cells. Table 3 then compares those clean inputs with the paired 8% boundary-removal inputs. This supports the measured robustness comparison without implying improvement over the clean condition.

Method / variantRemovalCoarseFineShapeNetPartPartNetEMean
PointCLIPV20%31.926.8223.6014.5819.23
PointCLIPV28%31.546.4623.0614.2118.82
PartSLIP++†^{\dagger}0%1.290.6317.241.335.12
PartSLIP++†^{\dagger}8%1.060.3716.911.124.87
Find3D0%31.1512.7623.3516.9021.04
Find3D8%30.6812.3422.8116.4520.57
CoSMo3D0%47.3024.2535.8517.8031.30
CoSMo3D8%46.4623.3134.3817.9430.52
DGV-SFR0%52.3629.1838.0820.7435.09
DGV-SFR8%51.3828.1437.1120.4634.27

Variant note: The †^{\dagger} marker indicates that PartSLIP++ uses its standard variant for Coarse/Fine and its dataset-specific variant for ShapeNetPart/PartNetE in both conditions. The object counts are 1,692, 1,692, 2,874, and 1,906; realized deletion rates are 7.98%, 7.98%, 8.01%, and 7.97%, respectively.

The four-benchmark mean decreases from 0% to 8% in the paired rows; DGV-SFR changes from 35.09 to 34.27. At 8% boundary removal, DGV-SFR obtains a mean mIoU of 34.27, compared with 30.52 for CoSMo3D. This is the protocol-aligned cross-method comparison; PointCLIPV2, PartSLIP++, and Find3D are adapted stress-test references and do not support a protocol-matched ranking. The result does not imply an improvement over DGV-SFR’s no-removal condition. CoSMo3D’s PartNetE score increases slightly, from 17.80 to 17.94. Removed points remain in the full-ground-truth evaluation and receive labels through nearest-neighbor lifting. Thus, removing an ambiguous boundary observation can change its lifted label and may occasionally replace an incorrect prediction with a correct neighboring prediction. This is a plausible effect of test-time masking and recovery, not evidence that removing points improves the model; we do not claim to have isolated this mechanism.

The PartSLIP++ rows use its standard variant for Coarse/Fine and its dataset-specific †^{\dagger} variant for ShapeNetPart/PartNetE, as indicated by the dagger in the table. The latter is trained with PartNetE data, unlike the other listed methods, so its PartNetE score is in-domain rather than zero-shot; this does not account for its ShapeNetPart score. The variant and training-data distinctions are described in Sec. 4.2, Table 1, and Appendix E.2.

The controls either replace adaptive Dirichlet-process weights with a fixed 16-component mixture or set training-time Wiener noise to zero, holding other settings fixed. Control values are means and standard deviations over three seeds; the full-model row reports Table 1 point estimates, not a matched three-seed result.

VariantCoarseFineShapeNetPartPartNetEOverall
Finite mixture52.067±0.35152.067\pm0.35129.133±0.35129.133\pm0.35134.767±0.35134.767\pm0.35120.900±0.20020.900\pm0.20034.217±0.31334.217\pm0.313
Deterministic training55.100±0.30055.100\pm0.30031.400±0.30031.400\pm0.30036.600±0.30036.600\pm0.30021.300±0.30021.300\pm0.30036.100±0.30036.100\pm0.300
Full DGV-SFR (Table 1)55.84032.29537.46021.70336.824

The finite-mixture mean (34.217) matches the posterior-surface ablation in Table 5 (34.22). Deterministic training gives 36.10 versus the full-model point estimate 36.824; this is descriptive because the full-model value is not a matched three-seed estimate. Neither control measures an independent path divergence. We have added their definitions and seed summaries to Appendix F.4.

Dear Reviewer HLQC,

We hope you are doing well. Since the discussion period will end in approximately one day, we would like to briefly follow up on our detailed response to your technical questions.

We clarified the variational formulation by writing the generative and variational path transitions explicitly and showing why their path log-ratio is zero. We also explained that the default one-step inference starts from the observed state, making the anchor force zero at that step, and restricted the density-ascent result to the continuous-time score component rather than presenting it as a universal finite-step boundary guarantee. On the experimental side, we specified the shared checkpoint and supervision protocol, clarified the aggregation difference between the boundary-missing 0% result and the main-table overall score, and explained that the stress test changes only the test point cloud without retraining.

We hope these clarifications resolve the remaining concerns about the ELBO, inference dynamics, and evaluation controls. We would appreciate your consideration of the complete response before the discussion closes.

Thank you for your careful technical review.

Best regards, Authors

Reviewer_nhH1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that point–text matching for open-vocabulary 3D part segmentation carries low-density outliers and mixed boundary responses into its predictions, and proposes DGV-SFR to recover the semantic field before an unchanged masked matching step. Each recovery block first moves point features toward a boundary-aware weighted mean of their k nearest neighbors, then fits a truncated Dirichlet-process mixture and moves points along the score of its posterior predictive density, weighted by uncertainty and constrained by a boundary projection and an anchor. The paper reports +4.1 mIoU over an internal direct-matching baseline and higher mIoU than CoSMo3D's reported numbers in every cell on four benchmarks.

Critical strengths
  • The framing is clean. prompts, masks, and the matching rule are untouched, so the module could plug into existing pipelines
  • Gains over CoSMo3D are consistent across all benchmarks, including the fine-grained split
  • The supporting experiments are broad, including ablations, calibration and boundary-leakage metrics, three seeds, and sensitivity sweeps
Critical weaknesses

1. Underspecified setup. The paper never states the backbone, text encoder, or training data. The module is trained with ground-truth part labels but compared against zero-shot and differently trained baselines. There is no held-out-part or cross-dataset test of the open-vocabulary claim.

2. Readability. The manuscript is hard to follow. A 29-row notation table and a full variational/SDE formalism describe what, at inference, amounts to a gated neighbor average followed by one gradient step on a fitted mixture density, and the intuition gets lost.

3. Reproducibility. Many hyperparameters and the q-inference procedure are unreported.

Questions for authors

What is the training data, and are the test parts and categories seen during training?

5 comments

Dear Reviewer nhH1,

Thank you for your assessment. We address each weakness and your question below, with the relevant evidence and manuscript locations.

[W1] Training setup and the scope of open-vocabulary generalization

Density-Guided Variational Semantic Field Recovery (DGV-SFR) trains its recovery modules on 3DCompat200 using point-level part labels for masked point–text segmentation and neighboring-label differences for boundary supervision; neither label is available at inference. The CoSMo3D Point Transformer feature extractor and SigLIP Base Patch16-224 text encoder are frozen, while recovery components are trainable. Our direct-matching baseline uses the same features, prompts, masks, and evaluation implementation, isolating the contribution of recovery: mean intersection-over-union (mIoU) rises from 32.72 to 36.82 (a 4.10-point gain; Sec. 4.4, Table 4). Literature baselines retain their reported protocols and are not matched-supervision comparisons (Sec. 4.1, Table 1; Appendix E.2).

For category transfer, the held-out 3DCompat200 split contains 140/30/30 train/validation/held-out categories and 18,742/3,811/4,206 objects (Sec. 4.2, Table 2; Appendix E.4). Held-out categories are excluded from fitting and model selection. The table reports category-macro mean intersection-over-union (mIoU, %), as mean ±\pm standard deviation across seeds 17, 42, and 73:

VariantCoarseFine
Direct matching49.00±0.3049.00\pm0.3024.97±0.2524.97\pm0.25
Denoising only50.20±0.2050.20\pm0.2026.43±0.2126.43\pm0.21
Full DGV-SFR51.13±0.1551.13\pm0.1527.53±0.2127.53\pm0.21

DGV-SFR gains 2.35 points over direct matching, averaged across Coarse and Fine; denoising-only results lie between them. This split holds out categories, not part names. It supports category transfer within 3DCompat200, but not part-disjoint or cross-dataset generalization. We have added the split definition in Sec. 4.2 and Appendix E.4.

[W2] Readability and operational interpretation

At inference, two recovery modules precede the unchanged masked point–text matching rule. Density-Guided Semantic Denoising (Sec. 3.2, Eqs. (2)–(3)) moves unreliable points toward prompt-consistent local modes using geometry, feature similarity, prompt responses, and boundary barriers; a reliability gate limits each update. Variational Dirichlet–Wiener Correction (Sec. 3.3, Eqs. (4)–(9)) is one unified module: a truncated Dirichlet process defines the posterior semantic surface, and Wiener score dynamics correct uncertain states on it under boundary projection and anchoring. The recovered features then use the original prompts and matching rule (Sec. 3.1, Fig. 2).

We have introduced this sequence before the notation and defined symbols at first use. Equation (9) establishes local density ascent only for the continuous-time score component; it does not guarantee boundary preservation by the full finite-step update (Appendix C.5).

[W3] Hyperparameters and variational inference

The default configuration and training procedure are:

SettingValueSettingValue
Recovery blocks LL4Neighbors kk12
Truncation level RR16Denoising strength η\eta0.14
Score-flow strength γ\gamma0.18Training steps per block TtrainT_{\mathrm{train}}2
Training step size Δt\Delta t0.5Training noise νtrain\nu_{\mathrm{train}}0.006
Dirichlet-process concentration α\alpha1.2Anchor coefficient λ\lambda0.10
Boundary likelihood coefficient λbnd\lambda_{\mathrm{bnd}}0.08Projection ridge δ\delta10−310^{-3}
Inference steps per block1Inference noise νinf\nu_{\mathrm{inf}}0
Evidence lower bound weight1.0OptimizerAdamW
Learning rate2×10−52\times10^{-5}Weight decay10−410^{-4}
Batch size / epochs4 / 3

The variational posterior uses Beta stick distributions, Normal–inverse-Wishart component distributions, categorical assignments, and Gaussian Euler path transitions (Sec. 3.3; Appendix C.4). Its parameters, including assignment responsibilities and component statistics, are optimized by gradient ascent on the joint objective; point uncertainty is computed from assignment entropy and posterior covariance, not as a separate variational factor. Because the generative and variational path kernels coincide, their Kullback–Leibler divergence is zero; the objective has no separate path regularizer. The default values are given in Sec. 4.1, and the full training and checkpoint protocol is specified in Appendix E.3.

[Q1] Training data and overlap between evaluation and training categories/parts

The recovery modules use 3DCompat200 point labels for segmentation and boundary supervision during training; inference uses no ground-truth labels. Evaluation-object basenames have no exact overlap with the recovery-training manifest. Category/part-name overlap is 200/200 categories and 988/989 part names for Coarse/Fine, 9/16 and 13/41 for ShapeNetPart, and 26/45 and 28/40 for PartNetE. Thus, the standard benchmarks are not category- or part-disjoint. These are exact-name counts; semantic synonym overlap was not assessed. The separate held-out split supports only within-3DCompat200 category transfer (Sec. 4.2; Appendix E.4).

Dear Reviewer nhH1,

We hope this message finds you well. As the discussion period will end in approximately one day, we would like to kindly follow up on our response to your valuable feedback.

In our rebuttal, we clarified the supervision and inference protocols, including that point-level labels and neighboring-label boundary information are used only for training and that no ground-truth labels are provided at inference. We also explained the roles of the two recovery modules, provided the default hyperparameters and optimization procedure, and clarified that the Dirichlet--Wiener path has no separate nonzero KL contribution. In addition, we reported the held-out category split and overlap analysis, while carefully limiting the generalization claim to category transfer within 3DCoMPaT200.

We hope these clarifications address your concerns about supervision, reproducibility, operational interpretation, and generalization. We would be grateful if you could take our detailed response into consideration during the remaining discussion period.

Thank you again for your time and thoughtful feedback.

Best regards, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.