acceptodds
Under review as a conference paper at ICLR 2027

Unity in Diversity: Low-Rank Multi-Relational Test-Time Adaptation for 3D Point Cloud Recognition

Abstract

Vision-language models have achieved strong zero-shot performance in 3D recognition, but their robustness degrades under point cloud corruptions and test-time distribution shifts. Existing point cloud test-time adaptation methods rely heavily on a single relation view, which can be unreliable under diverse corruptions. Thus, the paper highlights the global relational structure among unlabeled target samples. Specifically, we find that corrupted target relations retain compact low-rank structure, which indicates that low-rank modeling can be treated as a structural prior and a complementary view to the raw relation. Besides, when geometry is distorted, feature-space relations can be further captured by semantic relations. Raw, Low-rank, and Semantic relations together could formulate a comprehensive global multi-relational structure. Based on these observations, we propose LoReTTA, a training-free multi-relational target-set test-time adaptation framework that exploits target-set structure without updating the pretrained backbone. LoReTTA constructs multiple relation views from frozen target features and zero-shot predictions, models target dependencies through self-representation, and jointly regularizes the resulting views with a lightweight relation-mode low-rank proximal update. The refined relations are then fused into a sparse consensus graph for residual logit propagation. Experiments across multiple 3D vision-language backbones and corrupted and clean benchmarks demonstrate improved zero-shot robustness and generalization without updating the pretrained backbone. The source code will be publicly released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Point Cloud Recognition, Test-Time Adaptation, Low-Rank Representation, Multi-Relational Learning, Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_zzPkOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This article proposes LoReTTA, a training-free test-time adaptation method for 3D point cloud vision-language models. The authors observed that even when point clouds are subjected to different types of corruptions, the target test set still retains a certain low-rank structure, and the raw feature relations, low-rank relations, and semantic prediction relations can provide complementary information. Based on this, LoReTTA constructs a multi-relational view, models target sample dependencies through self-representation, and uses relation-mode low-rank consensus recovery to fuse different relations. Finally, prediction refinement is performed through a sparse consensus graph. The authors conducted experiments on ModelNet-C, ShapeNet-C, ScanObjectNN-C, and multiple open 3D benchmarks, and validated the effectiveness and robustness of the method on multiple 3D vision-language backbones.

Critical strengths
  1. Proposed a new perspective of using target set relational structure for test-time adaptation. Unlike traditional TTA methods that rely on a single feature or prediction information, LoReTTA simultaneously utilizes three types of relations: raw, low-rank, and semantic, and fuses them through low-rank consistency constraints, making the method design somewhat reasonable. 
  2. The method does not require updating the backbone and has good practical application potential. The adaptation method that avoids parameter updates has practical significance for large-scale pre-trained 3D vision-language models. The experiment also covers multiple backbones and various corruption settings, and the verification is relatively sufficient.
Critical weaknesses
  1. The experiment mainly focuses on artificially constructed corruption benchmarks, with insufficient validation of real domain shifts. The ModelNet-C, ShapeNet-C and other datasets mainly construct test distribution changes by artificially adding noise, dropout, rotation and other methods, which still differ from sensor shift and cross-dataset changes in real scenarios. Therefore, current experiments have provided more evidence of corruption robustness, while the generalization ability of actual test-time distribution shifts still lacks sufficient validation. 
  2. The method relies on modeling the relationship of the entire target set, and its applicability in online scenarios is not yet clear. LoReTTA needs to construct a relation matrix within the target set, so it is assumed that a batch of test samples can be accessed. For streaming or small batch testing scenarios, further clarification is needed to determine whether the method is still effective.
Questions for authors
  1. LoReTTA relies on target set-level relationship modeling. How will the performance of the method change if only a single test sample or a small batch can be obtained and the complete target distribution cannot be accessed? 
  2. The current experiment is mainly based on synthetic corruption benchmarks. Has the author tested domain shifts across datasets or closer to real-world scenarios to validate the effectiveness of the method for actual distribution changes? 
  3. Does the advantage of LoReTTA over existing training-free adaptation methods (such as cache-based or prompt-based VLM adaptation) mainly come from multi-relational modeling or from using larger-scale target set information? Can further fairness analysis be conducted?

Regarding the second question, it is not unique to you. It can be said that this is a problem that newcomers will encounter when using benchmarks from previous work. However, it is still hoped that the model can be evaluated or validated on real datasets, which is crucial for its practicality. Of course, as far as the current submitted work is concerned, it is already considered solid.

Reviewer_bnxXOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LoReTTA, a training-free TTA framework for robust 3D point cloud recognition under distribution shifts. It exploits complementary Raw, Low-rank, and Semantic relations among unlabeled target samples, models their dependencies through self-representation, and performs relation-mode low-rank consensus recovery before refining predictions via a sparse graph. The pretrained 3D VLM remains fully frozen throughout adaptation. Experiments on ULIP, ULIP-2, and Uni3D across multiple corrupted and clean benchmarks demonstrate improved robustness and competitive generalization.

Critical strengths

1. The paper identifies a meaningful structural limitation of existing point cloud TTA methods

The observation that a single relational view may become unreliable under corruption is well motivated. Raw feature relations, low-rank structural relations, and semantic prediction relations exhibit different failure modes and provide complementary neighborhood information to some extent. The analyses based on same-class purity, Top-10 overlap, and “rescue” rates provide empirical support for the proposed multi-relational formulation.

2. The method is training-free and keeps the pretrained 3D-VLM backbone fully frozen

This property is practically attractive for large pretrained 3D VLMs. LoReTTA does not require backpropagation, parameter updates, or task-specific retraining. Instead, adaptation is performed entirely through relational modeling over the target set and logit refinement.

3. The relation-mode low-rank update is computationally simple and lightweight

After stacking the three relation views into an N2×3N^2 \times 3 matrix, the nuclear-norm proximal optimization admits a closed-form solution via singular value thresholding. Since only three relation views are involved, the update operates on at most three singular values. The appendix further shows that SVT achieves the same downstream accuracy as iterative ADMM while requiring less solver time.

4. The experimental evaluation is comprehensive and includes several useful diagnostic analyses

The paper evaluates three different 3D VLM backbones and covers multiple corruption benchmarks, clean-set generalization, component ablations, fusion strategies, retained-rank sensitivity, target-set size, computational overhead, relation visualizations, and additional ScanObjectNN-C settings. These experiments provide a relatively complete picture of when the method is effective and where its gains become weaker.

Critical weaknesses

1. The comparison protocol is not fully matched in terms of target-set access

LoReTTA is a transductive target-set method that explicitly constructs dense N×NN \times N relation matrices over a large collection of unlabeled target samples, whereas some competing methods are evaluated with batch size = 1 or under online protocols. This means that LoReTTA has access to substantially more test-time context than single-sample or streaming-cache methods. Therefore, part of its performance gain may come from a stronger test-time information-access assumption rather than solely from the proposed low-rank multi-relational modeling. A comparison under matched target-set access would make the experimental conclusions more convincing.

2. The method does not consistently outperform competing approaches on the more challenging ScanObjectNN-C settings

On ScanObjectNN-C Obj Hardest, LoReTTA achieves the best performance with ULIP and Uni3D, but performs clearly worse than TDA with ULIP-2. Similar inconsistencies also appear on Obj Only and Obj BG, where TDA outperforms LoReTTA under some backbone/setting combinations. The paper attributes this behavior to severe corruptions weakening the reliability of relational structures, but this also reveals an important limitation of the method: when neighborhood structures are heavily distorted, the core assumption underlying relational refinement may no longer hold.

3. The method strongly depends on a fixed global target-set structure and has quadratic memory complexity

LoReTTA requires dense N×NN \times N relation matrices, resulting in O(N2)O(N^2) memory complexity. The authors also acknowledge this limitation in the scalability analysis. Although the overhead remains manageable for ModelNet-C with N=2468N=2468, it is unclear whether the method can scale effectively to substantially larger open-world target sets. This issue is particularly important given that the method is positioned as a practical test-time adaptation framework for 3D recognition.

4. The novelty mainly lies in combining existing techniques rather than introducing a fundamentally new adaptation principle

The core components of LoReTTA, including pairwise similarity, self-representation, low-rank modeling, nuclear-norm proximal optimization, graph construction, and label/logit propagation, are all well-established techniques. The primary contribution lies in organizing these components into a multi-relational, training-free TTA pipeline for 3D VLMs. While this integration is useful, the paper should more clearly articulate the conceptual novelty beyond existing low-rank relational learning and graph-based prediction refinement.

5. The Semantic relation may propagate incorrect zero-shot predictions, but its reliability is not explicitly controlled

The Semantic view is constructed directly from softmax-normalized zero-shot logits. Under severe corruption, these predictions may become highly unreliable, and the resulting semantic affinities may reinforce incorrect class neighborhoods. Although the paper acknowledges this risk and shows that the Semantic relation can provide complementary “rescue” effects, the current framework does not include confidence-aware gating, uncertainty weighting, or any explicit mechanism to suppress the Semantic view when predictions are unreliable.

6. The fixed hyperparameter setting may hide dataset-specific sensitivity

The paper uses a single configuration across all backbones and datasets, including r=20r=20, λ=0.1\lambda=0.1, β=0.05\beta=0.05, k=10k=10, and fixed fusion weights (0.1,1.0,0.01)(0.1, 1.0, 0.01). Using a unified configuration is a strength in terms of reducing dataset-specific tuning, but it also raises the question of whether these values are close to optimal or simply happen to work well for the reported target-set scales. In particular, the very small weight assigned to the Semantic relation appears somewhat inconsistent with the paper’s emphasis on it as an important complementary relation view.

Questions for authors

1. How much of LoReTTA’s performance gain comes from access to the full unlabeled target set?

Could the authors provide a matched comparison in which other training-free baselines are also allowed to access the same target-set batch? Alternatively, could LoReTTA be evaluated under online or mini-batch target access? Such experiments would help disentangle the contribution of the multi-relational formulation from the advantage of stronger transductive information access.

2. Could the authors further analyze the failure cases on ScanObjectNN-C, particularly with ULIP-2?

For example, under Drop-L and Jitter corruption, how do same-class relation purity, neighborhood overlap, and consensus quality differ from those on ModelNet-C? Such an analysis would directly validate the authors’ explanation that severe geometric corruption can destroy relational reliability.

3. How scalable is LoReTTA when the number of target samples grows beyond a few thousand?

Since the method requires dense N×NN \times N relation matrices, could the authors evaluate approximate kNN relations, blockwise processing, Nyström approximation, or sparse self-representation to demonstrate scalability to larger target sets?

4. How robust is the Semantic relation to incorrect zero-shot predictions?

Could the authors evaluate confidence-aware semantic weighting, entropy-based filtering, or selectively disabling the Semantic view for low-confidence samples? This would help determine whether the Semantic relation remains reliable under severe distribution shifts.

5. Could the authors more clearly clarify the conceptual novelty over classical low-rank multi-view learning and graph-based label propagation?

A stronger classical multi-view low-rank baseline using exactly the same three relation views would help determine whether the observed gains mainly come from the proposed relation-mode formulation or from standard multi-view fusion combined with graph-based propagation.

Reviewer_QbAGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LoReTTA, a training-free target-set adaptation framework for 3D point cloud recognition with frozen vision-language models. It constructs raw-feature, low-rank, and prediction-based relations, applies ridge self-representation and relation-mode singular value thresholding, and fuses the resulting relations for residual logit propagation. Experiments across three backbones and multiple benchmarks show broad improvements over zero-shot predictions.

Critical strengths
  1. The method outperforms the compared approaches in most settings across multiple datasets and backbone networks. The results support its applicability across these configurations.
  2. The method combines complementary views of sample relationships through relation fusion and prediction refinement, without updating the backbone. Ablations support the usefulness of multiview information. The authors also acknowledge the method’s performance limitations and quadratic memory requirements.
Critical weaknesses
  1. Differences in target-data access make the gains difficult to attribute. Section 4.2 uses target-set adaptation for LoReTTA, while the comparison methods process individual samples or follow their original online protocols. Target-set adaptation is a legitimate setting, but the current comparisons leave open how much of the improvement comes from relational modeling and how much comes from jointly accessing target samples. A graph-based inference or output-refinement baseline using the same target sets and frozen features would address this concern. I would consider raising my score if the authors clarify the comparison protocols and show that the proposed modeling provides gains beyond those from joint target-set access.
  2. The experiments do not sufficiently isolate the effect of relation-mode regularization. Figure 5 shows improved same-class relation mass, and the ablations support the overall approach. However, neither clearly establishes how singular-value shrinkage itself changes relation quality. Linear dependence across views does not necessarily imply semantic correctness: views can agree on incorrect relations, and information specific to one view can still be useful. A comparison immediately before and after the proximal update would help determine whether it recovers reliable consensus structure or mainly changes the relative contributions of the views.
Questions for authors
  1. How does LoReTTA compare with a graph-based inference or output-refinement baseline, such as LAME, when both use the same target sets and frozen features? This would help separate the contribution of relational modeling from the benefit of joint target-set access.
  2. Is there direct evidence that relation-mode singular-value shrinkage removes incorrect relations while preserving useful information specific to individual views? In particular, does the proximal update increase the proportion of semantically correct neighbors and reduce incorrect relations shared across views?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.