- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper looks at false negatives in contrastive pretraining for point clouds. In the standard NT-Xent loss, the more similar a negative is to the anchor, the harder it is pushed away. If that negative is really from the same class, this hurts. The authors propose ReCoR2, which multiplies each candidate in the NT-Xent denominator by a weight that decreases with similarity (a softmax over negative similarities). The result is that similar candidates are pushed away weakly and dissimilar ones strongly. Because similarities are not reliable early in training, the method first trains with NT-Xent and then switches to ReCoR2.
The paper gives a gradient analysis. Under three assumptions about how false and true negatives are separated in feature space, the gradient that a noisy false negative receives under ReCoR2 is bounded below the corresponding NT-Xent gradient. The loss is used inside a CrossPoint-style 3D-2D framework, extended with attention-based fusion of two point cloud views and a Chamfer-based geometry regressor. It is evaluated with linear and few-shot classification on ModelNet40 and ScanObjectNN and part segmentation on ShapeNetPart, with PointNet and DGCNN backbones. Small ablations cover the loss terms, the two-stage schedule, the choice of weighting function, and an injected false-negative setting.
Critical strengths
S1. A simple, clearly motivated change. The method is a small change to a standard loss and is easy to add to existing pipelines. The gradient analysis in Section 3.2 makes the motivation concrete: NT-Xent pushes similar negatives harder, and ReCoR2 reverses this. The paper is honest that this only makes sense once the feature space is informative, and builds the two-stage schedule around that.
S2. The key assumption is checked. Before using similarity as a weight, the paper tests on the Stage I features whether higher similarity really means a higher chance of the same class (Figure 3). It also adds a controlled experiment with injected false negatives (Appendix B.2). The method gives small but consistent gains over the reported baselines on both backbones and three datasets.
Critical weaknesses
W1. What the loss actually does is different from how it is presented. This is my biggest concern. The weights are chosen so that they exactly cancel the exponential terms in the denominator. After simplification (Eq. 8), the loss is the positive similarity plus a log-sum-exp over negative similarities. This is a soft minimum: the gradient goes almost entirely to the most dissimilar candidates, which are already far from the anchor. With the small temperature used, nearby negatives, both false and true ones, get almost no gradient. So the loss is not a mild reweighting of NT-Xent. It removes hard-negative pressure completely and focuses on easy negatives. This explains why the loss fails when used from the start (Table 4 shows it is then worse than plain CrossPoint), and why it only works as a short fine-tuning stage after NT-Xent.
The theory has the same issue. The theorem shows that a high-similarity sample gets a smaller gradient under ReCoR2. This is true by construction of the weights. It also assumes that every true negative lies below a similarity threshold (Assumption 2), which means it assumes there are no hard negatives. Under the same analysis, a true negative that is close to the anchor would also be ignored. That is the case where contrastive learning needs the gradient most, and the paper does not discuss it. The "maximum likelihood" framing is also not backed by a likelihood. No model or likelihood is written down, and the decreasing weight is a design choice, not a derivation.
W2. The gains are small and not clearly due to the loss. The reported improvements over the strongest baseline are small on most benchmarks. Part segmentation is a tie. The linear and segmentation results have no standard deviations, and the few-shot gaps are smaller than the reported standard deviations. More importantly, the comparison is not controlled:
- The framework is not the same as the baselines. It adds attention-based fusion of the two point cloud views and a geometry regressor with a Chamfer loss. The method also trains longer in total (NT-Xent stage plus ReCoR2 stage). The key control is missing: the same framework, trained for the same total epochs with NT-Xent only. The injected false-negative table suggests that, inside this framework, the gap between NT-Xent and ReCoR2 is small.
- There is no comparison with existing false-negative methods (debiased contrastive learning, hard-negative sampling, incremental false-negative detection), even though they are cited in Section 2.2. These are the most relevant baselines for this paper.
- The title and theory are about robustness to sampling noise, but there is no robustness experiment (for example, varying point density, jitter, or a corrupted benchmark such as ModelNet-C).
- Baseline numbers are copied from earlier papers, and masked-modeling methods discussed in related work are left out of the tables.
Questions for authors
Q1. Given Eq. 8, the gradient on each negative is proportional to exp(-s/tau), so it concentrates on the least similar candidates. Can you show how the gradient is spread across negatives during Stage II? How does ReCoR2 treat a true negative that is very similar to the anchor (a hard negative of a different class)?
Q2. Can you report the same framework (with view fusion and the geometry regressor) trained with NT-Xent only for the same total number of epochs as Stage I plus Stage II? Can you also report CrossPoint trained for the same total epochs? If ReCoR2 still gives a clear gain over these, my main concern in W2 would be resolved.
Q3. How does ReCoR2 compare with debiased contrastive loss, hard-negative sampling, and incremental false-negative detection, applied in the same framework and with the same two-stage schedule?
Q4. The paper claims robustness to sampling noise. Can you add an experiment that tests this directly, such as accuracy under varying point density, jitter, or a corrupted point cloud benchmark?
Q5. With batches of 16 objects drawn from 55 categories, how often does a batch contain same-class pairs? Also, in Section 3.1 the false negatives are described as noisy observations of the same surface. In the standard setup, other views of the same object are positives, not negatives. Which samples are the false negatives in practice?
Q6. Injected false negatives (Appendix B.2). How are false negatives injected? Why does the reweighted accuracy in this table differ from the main result under the same backbone and dataset?
Q7. Which likelihood is being maximized? If there is a probabilistic model behind the weights, please state it. Otherwise, I suggest describing the weights as a design choice.
Minor:
Q8. Please report standard deviations over several pretraining seeds for the linear classification, segmentation, and ablation results (Tables 1, 3, 4, 5). Many of the differences there are very small.
Q9. Table 1 has no PointNet result for CCPoint on ScanObjectNN, so the best result in that column is compared against an incomplete set. Please note this in the table.
Q10. Figure 4 says accuracy "exceeds the baseline" after the switch. Which baseline is meant? Is it NT-Xent continued for the same number of epochs?