- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper looks into depth estimation for transparent objects in autonomous laboratories, and puts forward Retrieval‑Augmented Transparent Depth Estimation (RATDE). The core thinking here is that lab environments are kind of a semi‑open world. Lots of lab glassware and equipment are standard items, so their corresponding CAD models might already live in some external object memory bank. That means we don’t have to rely entirely on the tricky visual signals from transparent objects themselves — those signals get messed up easily by refraction and reflection — to figure out object geometry. The authors also built the dataset LabSense‑Glass. For their actual experiments, they used nine simulated lab setups and 1,336 stereo samples. They also provide annotations like paired renderings for transparent / opaque versions, depth maps, normal maps, instance masks, and camera calibration data.
Critical strengths
Treating lab environments as a semi‑open world is a very reasonable angle. Existing CAD libraries really can supply more trustworthy shape priors than noisy RGB signals from transparent objects. Importantly, the method does not get told the ground‑truth CAD identity or object pose during inference; it has to retrieve and align CAD models from memory on its own. The connections between silhouette retrieval, Utonia point features, DINOv2 / MoGe‑2 descriptors, PnP pose fitting, local geometry tokens, SeeClear and LingBot‑Depth are easy enough to follow. I especially like how they separate global retrieved shape priors from pose‑aware local geometry; this is sensible design.
Critical weaknesses
- The paper needs to spell out exactly how they split training / validation / test data. Are splits done by picking random individual frames? Or do they keep whole scenes separate across splits? Do evaluation scenes never show up in training at all? Have target objects used in evaluation already appeared in data used to train the conditioning modules? Could the same object instance or nearby camera views leak across train and test sets? This matters a lot. The core setup assumes the exact matching CAD exists inside memory — that premise is fine. But if test images share scenes, assets or nearby camera trajectories with training data, then the measured results mainly show reconstruction performance within familiar scenes instead of true generalization.
- One super important baseline is missing: an image‑only baseline adapted for lab data. The authors first LoRA‑fine‑tune the detector and the image‑only SeeClear model, and only then train the point‑feature conditioning and local residual modules. However, the SeeClear baseline they compare against in Table 2 is the publicly released checkpoint without this lab‑specific LoRA tuning. The authors themselves admit the gap between plain SeeClear and full RATDE mixes two factors: domain adaptation gains plus improvements from CAD‑based conditioning. Therefore we really need results for lab‑adapted image‑only SeeClear + LingBot + SGBM. That means applying exactly the same LabSense‑Glass LoRA domain adaptation, but completely removing both CAD‑based conditioning branches. Running this baseline would clearly tell us how much improvement comes purely from lab‑domain fine‑tuning and how much actual benefit comes from CAD conditioning. Their existing Random‑CAD control experiment is useful, but it cannot fully substitute for this baseline. The full model was trained expecting normal retrieval‑conditioned inputs. Swapping in random CADs only during testing shifts the input distribution for conditioning.
- All main experiments run on just nine Isaac Sim lab environments. Even though the appendix lists ClearDepth as an external benchmark resource, the main text shows no results on outside real‑world or external benchmark datasets. We know simulation‑to‑real gaps can be huge for transparent‑object perception. Even some small‑scale sanity‑check experiments on real‑world data, or tests on external benchmarks, would make the paper’s conclusions much stronger. I won’t reject the paper purely for lacking large‑scale real‑world tests, since this CAD‑conditioned task requires CAD model associations that most existing datasets don’t provide.
- There are clear inconsistencies when describing LabSense‑Glass dataset size. Early manuscript sections make it sound like “231 assets / 222 scene entries” are part of LabSense‑Glass itself. But the appendix clarifies that the actual experimental LabSense‑Glass dataset only has 9 environments and 1,336 stereo samples. Moreover, only a subset out of those 231 assets act as transparent target objects in experiments.
- Equation (6) relies on and to convert normalized CAD coordinates back into real‑world metric geometry. Yet the appendix says nominal object dimensions are used “when available”. The authors should clarify: do every transparent‑target CAD model in their experiments have reliable real‑world size information? If some CAD assets lack nominal dimension metadata, how do they set values for , handle Equation (6), and fix the reference depth for those objects?
Questions for authors
- Can you give us the exact protocol you used for train / validation / test splitting, as brought up in Weakness point 1?
- Can you report results for that lab‑adapted image‑only SeeClear + LingBot + SGBM baseline mentioned in Weakness point 2?
- Would it be possible for you to run those small‑scale real‑world experiments discussed in Weakness point 3?
- Can you clear up confusion about LabSense‑Glass dataset scope from Weakness point 4?
- Please address the metric‑dimension issue raised in Weakness point 5.