acceptodds
Under review as a conference paper at ICLR 2027

Failure-Aware Adaptation for Transparent Objects in Feed-Forward 3D Reconstruction

Abstract

Feed-forward 3D foundation models reconstruct scene geometry from a set of in- put views without per-scene optimization, yet remain unreliable for transparent objects. We analyze these failures and find that transparent-object errors concen- trate among a small fraction of high-error predictions and separate into two asym- metric modes, overestimation and underestimation. Overestimation is primar- ily associated with background leakage, whereas underestimation concentrates around concave structures, revealing distinct geometric characteristics between the two failure modes. Motivated by this asymmetry, we introduce a failure- aware parameter-efficient adaptation framework with two complementary mod- ules. Background-aware LoRA selectively adapts regions associated with back- ground leakage using background geometric cues. Curvature-Informed Corrector infers error directions from local geometric cues and corrects overestimation and underestimation through separate correction paths. Our method achieves state-of- the-art performance on depth estimation benchmarks for transparent objects while consistently improving reconstruction accuracy across different feed-forward 3D foundation models.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D reconstruction, depth estimation, transparent object, low rank adaptation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jEGcOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies depth estimation failures of feed-forward 3D foundation models on transparent objects. It identifies concentrated high-error predictions and an asymmetry between overestimation associated with background leakage and underestimation associated with flattened concave structures. The proposed adaptation framework combines Background-aware LoRA, which gates additional low-rank updates using geometric background affinity, with a Curvature-Informed Corrector (CIC), which predicts error directions from local geometry and routes correction through two branches. Evaluation covers three backbones (VGGT, π³, and DA3), ClearPose training and testing, and transfer to ClearGrasp and TRansPose. The paper reports substantial depth improvements and provides parameter-matched ablations, directional analyses, and failure cases.

Critical strengths
  1. Substantial empirical improvement with a small adaptation budget. On ClearPose, the VGGT-based method reduces RMSE from 75.8 mm for LoRA r16 to 40.6 mm while using approximately 5.9M trainable parameters (Table 1). Evaluation across three backbones and two additional datasets broadens the evidence, despite some transfer failures. The magnitude of the in-domain improvement makes the approach potentially useful for transparent-object perception.

  2. A useful failure analysis supported by targeted evidence for background-aware adaptation. The paper connects error direction to distinct geometric cues and translates this observation into selective adaptation. At a matched parameter budget, BG-LoRA improves RMSE from 77.9 mm for LoRA r12 to 60.1 mm; removing leakage selection gives 76.5 mm (Tables 4 and 15). These controls provide meaningful evidence that background-aware selection contributes beyond additional parameters.

Critical weaknesses
  1. Transfer gains lack a matched LoRA baseline. Table 3 compares the full method with zero-shot models, without ordinary LoRA trained on the same data. The additional cross-dataset benefit of the proposed modules over ordinary adaptation therefore remains unclear.

  2. The specific contribution of directional routing is insufficiently validated. Routing reduces RMSE from 41.5 to 40.6 mm (Table 15), but the consistency of this gain across scenes and its benefit on the targeted overestimated and underestimated regions are not established. This limits the evidence for a central design choice.

Questions for authors
  1. Validation request — cross-dataset advantage. How does the full method compare with ordinary LoRA trained on the same ClearPose data and evaluated under the same transfer protocol? Results for VGGT on ClearGrasp and TRansPose would directly address this concern; the DA3/ClearGrasp comparison would also help explain the reported transfer limitation.
3 comments
AuthorsOpenReview ↗

Response to Reviewer jEGc

We thank the reviewer for the careful reading and for identifying the two places where the evidence was thin. Both concerns are addressed with new experiments below.

Transfer with a matched LoRA baseline

We added LoRA r16, trained on the same ClearPose data, to each cell of Table 3 and evaluated it with the same harness and protocol as our method. LoRA r16 is the baseline of Table 1, and it is trained for each of the three backbones with the same recipe.

Table R1. Table 3 of the main manuscript with matched LoRA baselines. RMSE in mm, AbsRel, and δ1.01 in percent. ClearPose and TRansPose use windows of six views, per frame affine alignment on all valid pixels, the transparent region, and the frame mean. ClearGrasp uses single images under the SeeClear protocol with affine alignment in the native output space of the model, on 113 images. Bold marks the best value within each backbone.

BackboneMethodClearPose RMSEClearPose AbsRelClearPose δ1.01TRansPose RMSETRansPose AbsRelTRansPose δ1.01ClearGrasp RMSEClearGrasp AbsRelClearGrasp δ1.01
VGGTzero-shot155.40.10537.276.50.12069.846.40.07478.0
VGGTLoRA r1675.80.045226.061.60.082611.526.40.041816.4
VGGTOurs40.60.028936.450.30.079012.418.80.028625.5
π³zero-shot151.60.10227.064.70.094011.239.70.061710.5
π³LoRA r1673.60.041325.358.20.084610.722.70.035015.1
π³Ours44.90.030733.557.70.092410.617.10.025332.3
DA3zero-shot167.50.09389.387.20.12908.323.90.038119.6
DA3LoRA r1668.30.041423.561.80.091210.823.90.037820.1
DA3Ours39.90.029034.855.40.084010.325.50.039415.3

LoRA r16 is matched to our method in trainable parameter budget, which makes it the appropriate reference for the additional gain of our modules. LoRA transfers partially on its own, and our modules add a further gain on top of it in most settings. The gain is smallest where the zero-shot error of the backbone is already low, as on DA3 with ClearGrasp, which leaves little for a correction learned on ClearPose to remove.

Consistency of the directional routing gain across scenes and its effect on the targeted regions

We retrained the corrector over four seeds for our method and for each control, keeping stage 1 fixed, and scored each configuration per scene and per error direction. The controls remove the sign split with matched parameters, shuffle the routing while keeping the fraction of pixels sent to each path, and remove the curvature features from the direction classifier.

Table R2. ClearPose test, VGGT. RMSE in mm, mean ± sample standard deviation over four seeds. Over and under denote the RMSE on overestimated and underestimated pixels under the protocol of Table 17 of the supplementary material.

ConfigurationseedsRMSEoverunder
Ours440.53 ± 0.2556.44 ± 2.3746.79 ± 0.71
w/o sign split441.66 ± 0.2952.99 ± 1.5050.02 ± 0.46
shuffled routing442.04 ± 0.8759.55 ± 4.8048.36 ± 0.54
w/o curvature features442.17 ± 0.7760.05 ± 4.5748.84 ± 0.58
Across seeds and scenes the gain is consistent. The seed range of our method does not overlap with that of any control, and our method is better on most of the 15 test scenes.

Removing the split, shuffling the routing, or removing the curvature features each raise the underestimation error by a similar amount.

These results strengthen the evidence for the paper’s contributions. I maintain my original recommendation of weak acceptance.

We are sincerely grateful for your thoughtful response and for your engagement throughout the discussion period. Should any concerns remain, we would greatly appreciate it if you could let us know, and we will make every effort to address them.

Reviewer_KWhuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper analyzes failure patterns of feed-forward 3D foundation models on transparent objects and proposes a framework to improve their reconstruction accuracy. To address background leakage, the authors introduce background-aware LoRA with object-region supervision. To address depth underestimation around concave structures, they propose a curvature-informed corrector that predicts error directions and applies separate correction branches.

Critical strengths
  • The paper provides a useful analysis of failure patterns in feed-forward 3D foundation models on transparent objects.
  • The proposed framework is evaluated across multiple backbones and datasets, demonstrating its broad applicability.
  • The method achieves substantial improvements over the evaluated baselines on several benchmarks.
Critical weaknesses
  • The paper states that GT masks are used for evaluation. But MODEST does not natively accept GT masks. It is unclear whether SeeClear and MODEST also use GT masks. Since both SeeClear and MODEST can run without GT masks, it's better for the authors to report results using predicted instance masks.
  • The reported metrics only evaluate transparent-object regions. It is unclear whether adaptation degrades accuracy on background or opaque regions. The author should also report results on non-transparent regions or the full image, alongside the original backbone and relevant baselines, to assess whether the improvements preserve overall scene-depth accuracy.
Questions for authors
  • Since the method repeatedly runs the same network to compute predictions and correction gates, please report end-to-end inference time and compare with SeeClear and MODEST under the same hardware and input settings.
  • Some minor mistakes in paper writing:
    • In Eq. (5), D^\hat D should be D^1\hat D_1 (as shown in Fig.3). D^\hat D is the final predicted depth.
    • In Figure 3, the hidden features enter the direction classifier. Appendix C describes its inputs as handcrafted features derived from D^1\hat D_1 and the instance masks.
    • Table 4 mainly compares additional LoRA ranks, while Table 1 already includes several adapter baselines. Replacing Table 4 with the more detailed component ablation in Appendix Table 15 would better highlight the contributions of BG-LoRA and CIC.
6 comments
AuthorsOpenReview ↗

Response to Reviewer KWhu, Q1. End-to-end inference time

End-to-end inference time

We timed each method end to end on a single TITAN RTX with batch size 1, on 20 ClearPose test windows of six views at 518 px after three warm up windows. Although our method is trained in stages, inference is a single pass through the pipeline. The frozen backbone is run once to obtain the affinity and the gate, the adapted backbone is then run once with BG-LoRA, and the corrector reuses the tokens of that second pass and only re-runs the depth head, so no stage is repeated and no cached intermediate is required at test time.

Table R1. End-to-end inference time on one TITAN RTX. Time per window and per frame in seconds, mean ± standard deviation over 20 windows. Peak memory in GiB.

Methodper windowper framepeak memory
VGGT zero-shot2.59 ± 0.010.4327.9
VGGT with LoRA r162.70 ± 0.010.4506.3
VGGT with Ours11.03 ± 1.181.83811.5
VGGT with SeeClear73.09 ± 6.6912.18215.2
MODEST0.39 ± 0.000.0660.9

The additional cost of our method over the frozen backbone comes from the second pass and from the affinity surface fits and classifier features. SeeClear is substantially slower, since it runs diffusion opacification on the object masks before the depth network. MODEST is a single image network at low resolution and is correspondingly fast, and Table 1 of the main manuscript gives its accuracy.

AuthorsOpenReview ↗

Response to Reviewer KWhu, W1. GT masks and results with predicted instance masks

Use of GT masks and results with predicted instance masks

The reviewer is right that this needs to be stated clearly, and we distinguish two uses of the mask.

For scoring, the methods in all tables, including SeeClear, MODEST, the PEFT baselines and ours, are scored inside the GT transparent object mask, which is the standard protocol for transparent object depth. MODEST and SeeClear are run as released without any mask input.

As an input, our method additionally takes the instance mask at training and inference. It defines the object and background regions of the affinity surface fits in Appendix B.1, the scope of the training loss, and the instance restriction of the direction classifier features in Appendix C.1, and at test time this mask is taken from the annotation. No ground truth depth is used at inference. This is an asymmetry in favor of our method, as the reviewer correctly identifies, and we quantify it below.

For the results with predicted masks, we ran the full pipeline with predicted masks as input, keeping all trained weights fixed, and scored inside the annotated mask so that each row is directly comparable to Table 1. The first predicted mask is the one MODEST itself produces with its segmentation head, so that feeding this mask to our method gives it the same mask information MODEST has. The second is Grounded-SAM2 [1], an open vocabulary detector that has not been trained on transparent objects.

Table R2. ClearPose test, per frame affine alignment, scored inside the annotated transparent mask. RMSE and MAE in mm. Bold marks the best value in each column.

MethodMask inputRMSE ↓AbsRel ↓MAE ↓δ1.01 ↑δ1.03 ↑δ1.05 ↑δ1.10 ↑δ1.25 ↑
VGGT zero-shotnone155.40.105386.77.2122.4138.9770.1893.25
MODEST †none101.00.074470.014.5037.0351.5673.8193.01
SeeClear with VGGTnone133.50.080869.79.2428.7548.3278.5896.19
LoRA r16none75.80.045238.426.0262.1377.5889.8497.72
OursGT40.60.028924.736.4273.2785.3894.3498.92
OursMODEST segmentation head56.60.032527.735.4371.9684.1893.5798.53
OursGrounded-SAM264.50.039433.724.7460.4780.2993.2498.53

With the mask of MODEST our method remains more accurate than the compared methods, and its accuracy at the tightest threshold is largely preserved. Even with a mask detector it remains more accurate than the compared methods, with a meaningful part of the gain retained in both settings.


[1] Ren, Tianhe, et al. "Grounded sam: Assembling open-world models for diverse visual tasks." arXiv preprint arXiv:2401.14159(2024).

AuthorsOpenReview ↗

Response to Reviewer KWhu, W2. Accuracy on non-transparent regions and the full image - 1

Accuracy on non-transparent regions and the full image

We re-scored each row of Tables 1 to 4 on the non-transparent region, defined as all valid ground truth pixels outside the GT transparent mask. The ClearPose test split carries no opaque object labels, so this region contains both background and unlabelled opaque objects.

In short, our adaptation does not degrade the non-transparent region. In the training domain it improves this region for the three backbones, by 8 to 14 percent in RMSE, and on ClearGrasp it improves it more than SeeClear does. The transparent specific baselines either leave the region unchanged by construction, as SeeClear does, or degrade it, as MODEST does. The exception is under transfer for two backbone and dataset pairs, which we report below.

Table R3. Table 1 of the main manuscript on the non-transparent region, ClearPose test. RMSE and MAE in mm.

MethodParamsRMSE ↓AbsRel ↓MAE ↓δ1.01 ↑δ1.03 ↑δ1.05 ↑δ1.10 ↑δ1.25 ↑
VGGT298.20.0855119.617.1146.7065.8884.9294.66
LoRA r166.1M294.50.0673107.134.5064.8376.6887.8295.74
DoRA6.4M297.20.0680108.733.6864.1875.9487.3895.54
AdaLoRA6.1M248.00.065289.829.7162.3777.1490.5797.10
LieRA r166.1M267.70.072599.724.1156.5872.8388.8296.28
KRAdapter11.1M249.50.065089.031.1463.5277.6190.7896.99
Ours5.9M257.90.064290.931.2165.8079.2190.3496.81
SeeClear294.70.0924125.314.4041.5260.6281.3593.97
MODEST †327.50.0910136.112.3737.1254.8677.3192.05
In the non-transparent region the PEFT methods are generally strong, since they distribute their capacity across the whole scene. Although our method is designed specifically for transparent objects, it remains comparable to them in this region and gives the lowest AbsRel among the compared methods. This is most evident in comparison with the transparent specific methods. SeeClear replaces only the object pixels before running the backbone, so its background remains essentially that of the backbone, and MODEST is less accurate than the frozen backbone outside the objects even within its own depth range.

Table R4. Table 2 of the main manuscript on the non-transparent region, ClearGrasp. Rows in italics are taken from the published manuscript of SeeClear.

MethodRMSE ↓AbsRel ↓MAE ↓δ1.01 ↑δ1.03 ↑δ1.05 ↑δ1.10 ↑δ1.25 ↑
VGGT19.90.020114.046.5382.7590.2996.5399.81
VGGT with Ours15.60.015010.557.7286.8392.6699.2399.94
VGGT with SeeClear19.40.019213.450.2384.6691.2096.3399.81
DA341.40.037527.932.9068.0480.0489.5297.88
DA3 with SeeClear39.40.035526.535.4069.5780.9390.2398.14
Depth4ToM45.560.04430.09——72.2385.55—
MODEST113.280.08759.09——73.5279.56—
D4RD25.710.02315.54——88.5394.44—
DKT †139.870.11372.04——74.3580.49—

Under transfer to ClearGrasp, our method improves the non-transparent region by about 22 percent without any training on that dataset, compared with 2 to 5 percent for SeeClear on either backbone, and is the best on the reported metrics.

Reviewer_RpTnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper analyzes transparent-object depth errors in feed-forward 3D models, associating overestimation with background leakage and underestimation with flattened concavities. It proposes Background-aware LoRA and a Curvature-Informed Corrector with direction-specific branches. Experiments cover three backbones and three datasets.

Critical strengths
  1. Substantial empirical improvements with a small adaptation budget. Improvements on ClearPose extend to all three backbones, and VGGT transfers to ClearGrasp and TRansPose without additional training. These results provide meaningful evidence that the overall adaptation strategy is useful to some extent.
  2. The directional-error analysis provides an interpretable motivation. Oracle routing, failure cases, and unsuccessful magnitude-recovery experiments help clarify the method’s capabilities and limitations.
Critical weaknesses
  1. Important implementation and evaluation details remain unresolved. Test-time instance-mask sources and multi-view sampling are unspecified.
  2. Evidence for direction-specific correction is limited: removing the sign split changes RMSE only from 40.6 to 41.5 mm, without multi-seed uncertainty estimates. Curvature-specific and shuffled-routing controls are missing. Furthermore, independently aligned depth errors do not establish consistent multi-view reconstruction, and MODEST is evaluated under a different alignment protocol.
Questions for authors
  1. What supplies the instance masks during training and inference on each dataset? Are any test annotations used as model inputs? If oracle masks are used, how much of the gain remains with predicted masks, and are comparison methods given equivalent mask information? Please also provide the training/validation scene lists, view counts, frame sampling, and window construction.
  2. Can you repeat the Table 15 sign-split ablation over multiple seeds, confirm parameter and training-budget matching for the final configuration, and add a shuffled-routing control? How does removing curvature while retaining the other classifier features affect routing accuracy and final depth error?
  3. Can you report MODEST under the same alignment used for the other methods? What is the precise aggregation of pixel, frame, and scene errors used in the main tables?
  4. Do the depth improvements persist under a shared sequence-level alignment, and is there evidence of improved cross-view consistency or fused geometry? If such evaluation is unavailable, would you narrow the reconstruction claim to the aligned depth-estimation setting actually evaluated?
5 comments
AuthorsOpenReview ↗

Response to Reviewer RpTn, Q1. Instance masks and data specification

What supplies the instance masks, and are test annotations used as model inputs?
Yes. The transparent object instance mask is an input to our method at both training and inference, and at test time it is taken from the dataset annotation, namely the instance labels of ClearPose and TRansPose and the connected components of the binary mask of ClearGrasp. The mask defines the object and background regions of the affinity surface fits in Appendix B.1, the scope of the training loss, and the instance restriction of the direction classifier features in Appendix C.1. No ground truth depth is used at inference. Scoring inside the GT mask is the standard protocol in transparent object depth estimation and is applied in the same way to all methods in all tables.

How much of the gain remains with predicted masks, and are comparison methods given equivalent mask information?
We ran the full pipeline with predicted masks as input, keeping all trained weights fixed, and scored inside the GT mask so that the rows below are directly comparable to Table 1. The first mask source is the segmentation head of MODEST, which is the mask MODEST itself produces and uses, so that this row gives our method the same mask information MODEST has. The second is Grounded-SAM2 [1], a generic open vocabulary detector that has not been trained on transparent objects.

Table R1. ClearPose test. per frame affine alignment, scored inside the GT transparent mask. RMSE and MAE in mm. Bold marks the best value in each column.

MethodMask inputRMSE ↓AbsRel ↓MAE ↓δ1.01 ↑δ1.03 ↑δ1.05 ↑δ1.10 ↑δ1.25 ↑
VGGT zero-shotnone155.40.105386.77.2122.4138.9770.1893.25
MODEST †none101.00.074470.014.5037.0351.5673.8193.01
SeeClear with VGGTnone133.50.080869.79.2428.7548.3278.5896.19
LoRA r16none75.80.045238.426.0262.1377.5889.8497.72
OursGT40.60.028924.736.4273.2785.3894.3498.92
OursMODEST segmentation head56.60.032527.735.4371.9684.1893.5798.53
OursGrounded-SAM264.50.039433.724.7460.4780.2993.2498.53

With the MODEST mask our method remains better than the compared methods on the reported metrics, and its accuracy at the tightest threshold is largely preserved. Even with a mask detector, our method remains more accurate than the compared methods on most metrics.

Training and validation scene lists, view counts, frame sampling, and window construction.
The ClearPose split is at the scene level, no scene is shared across splits, and every capture condition from set1 to set9 appears in both training and test. The scene lists are given in Table R2.

Table R2. ClearPose scene split. Entries are scene numbers within each set.

Splitset1set2set3set4set5set6set7set8set9scenesframes
train1, 3, 51, 3, 61, 3, 81, 5, 61, 5, 62, 5, 61, 3, 4, 51, 2, 3, 67, 8, 9, 10301,950
validation4114335414
test2, 4542, 32, 442, 64, 511, 12151,068

Each scene is cut into non-overlapping windows of 6 views in frame order, which yields 178 test windows, and the trailing frames of a scene that do not fill a window are dropped for all methods. All six views of a window are passed to the backbone jointly, and metrics are computed per frame. For TRansPose we use the Household subset with the camera cam_R, all nine sequences are held out as test data, and each sequence contributes 20 windows sampled at a stride of 12 frames, chosen so that the within window baseline matches ClearPose, giving 1,080 frames. For ClearGrasp we use the 113 real test images, each treated as a single view window. No ClearGrasp or TRansPose frame is used in training or validation. Scene lists and window indices will be released with the code.


[1] Ren, Tianhe, et al. "Grounded sam: Assembling open-world models for diverse visual tasks." arXiv preprint arXiv:2401.14159(2024).

AuthorsOpenReview ↗

Response to Reviewer RpTn, Q2. Validation of the sign split

Validation of direction-specific correction with multiple seeds, parameter matching, a shuffled-routing control, and curvature removal

We ran the four requested ablation studies. In each case only the corrector stage was retrained on the fixed stage 1 model, and all other hyperparameters were kept identical to the paper as given in Table 10. Table R3 collects the results and Table R4 gives the classifier accuracy.

Table R3. ClearPose test. RMSE in mm, mean ± sample standard deviation over four seeds. Over and under denote the RMSE on overestimated and underestimated pixels under the protocol of Table 17 of the supplementary material. The single path control is matched to our method in parameters and training budget, and in each ablation only the corrector stage is retrained on the fixed stage 1 model.

ConfigurationseedsRMSEoverunder
Ours440.53 ± 0.2556.44 ± 2.3746.79 ± 0.71
w/o sign split441.66 ± 0.2952.99 ± 1.5050.02 ± 0.46
shuffled routing442.04 ± 0.8759.55 ± 4.8048.36 ± 0.54
w/o curvature features442.17 ± 0.7760.05 ± 4.5748.84 ± 0.58

Table R3 shows that the gain of our method is robust. The single path control is matched in parameters and training budget. Shuffling the routing while keeping the fraction of pixels sent to each path, and removing the curvature features from the classifier, are both worse than removing the split altogether.

Removing curvature while retaining the other classifier features. We refit the logistic classifier on the seven features that do not involve curvature, predicted depth, gradient, Laplacian, local standard deviation, two edge distance features and instance area, and retrained the corrector.

Table R4. Classifier accuracy on the ClearPose test split. Routing ratio denotes the fraction of object pixels sent to the overestimation path.

ClassifierAUC, all object pixelsAUC, top 10% error pixelsRMSE
19 features as in the paper0.5810.82640.53 ± 0.25
7 features without curvature, τ_sign 0.450.5610.70842.29 ± 0.70
7 features without curvature, τ_sign matched (0.5135)0.5610.70842.18 ± 0.77
Without curvature the classifier loses much of its ability to separate the two directions on the high error pixels, and the final depth error increases even when the routing ratio is matched. The curvature features carry the largest fitted weights in Table 8, so this is the expected consequence of removing them.
AuthorsOpenReview ↗

Response to Reviewer RpTn, Q3. MODEST alignment and aggregation protocol

MODEST under the same alignment, and the aggregation used in the main tables

MODEST under the same alignment. Table 1 scores each row with a per frame affine fit of scale and shift on all valid pixels in the native output space of the model, except MODEST, which is scored on its native metric output without alignment and marked with †. Table R5 gives MODEST under both protocols.

Table R5. MODEST on the ClearPose test split, same frames and mask as Table 1 of the main manuscript. RMSE and MAE in mm.

AlignmentRMSE ↓AbsRel ↓MAE ↓δ1.01 ↑δ1.03 ↑δ1.05 ↑δ1.10 ↑δ1.25 ↑
none, native metric output, as in Table 1101.00.074470.014.5037.0351.5673.8193.01
per frame affine on all valid pixels, as in the other rows292.50.3521268.66.2017.4624.6936.8456.46

Under the common protocol MODEST degrades substantially, for a reason specific to the released model rather than to alignment in general. On ClearPose it predicts the background at 2 to 3 m as a nearly planar surface close to the camera, so a scale and shift fitted on the whole frame are dominated by that collapsed background and displace the object depths, which are in fact well scaled. We therefore score MODEST on its native output, as described in Appendix E. It is a metric scale model, and its own paper evaluates it in this way, following the depth completion convention of comparing predicted metric depth directly to ground truth and reporting a ClearPose RMSE of 0.123 m. Our unaligned reproduction of 101.0 mm is better than that reported value, so the † protocol does not understate MODEST. SeeClear with VGGT is a relative depth pipeline and is scored under the same per frame affine fit as the other rows.

Aggregation of pixel, frame, and scene errors. For all methods and tables, a scale and shift are first fitted by least squares on all valid pixels of each frame in the native output parameterisation of the model, which is depth for the point map backbones, and converted to depth. All metrics are then computed per frame on the pixels inside the GT transparent mask that carry valid ground truth. The reported value is the unweighted mean over the 1,068 test frames.

Comments on the paper

1 comment

We sincerely thank all reviewers for their constructive feedback. We have addressed each request with new experiments and clarifications, summarized below.

  1. Instance masks (Reviewer RpTn, KWhu): We clarified how masks are used, and report results with predicted masks (MODEST segmentation head, Grounded-SAM2). Our method still outperforms all baselines in RMSE.
  2. Evaluation protocol (Reviewer RpTn): We added the scene split, window construction, MODEST under the common alignment, and the exact error aggregation.
  3. Geometric evaluation (Reviewer RpTn): Under sequence-level alignment and on fused geometry, our method remains the best, roughly halving the Chamfer distance of the best baseline.
  4. Directional routing (Reviewer RpTn, jEGc): Multi-seed ablations with matched-parameter, shuffled-routing and no-curvature controls confirm a consistent gain.
  5. Transfer with matched LoRA (Reviewer jEGc): Our method provides further gains over a parameter-matched LoRA baseline under the same transfer protocol in most settings.
  6. Non-transparent regions (Reviewer KWhu): Our adaptation preserves or improves accuracy outside transparent objects.
  7. Inference time (Reviewer KWhu): We report end-to-end time and memory against SeeClear and MODEST on the same hardware.

We hope these responses have addressed the reviewers' concerns. Although the discussion period is ending soon, we remain committed to addressing any remaining concerns, and all of the above will be reflected in the main manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.