acceptodds
Under review as a conference paper at ICLR 2027

AVA-3D: Attribute-based Visual Arbitration for Unified 3D Representation Learning

Abstract

Multimodal pre-training for 3D point clouds has shown strong potential by aligning 3D shapes with corresponding images and language descriptions. However, existing methods often integrate multi-view visual information at the global or view level, overlooking heterogeneous visual attributes across viewpoints and their varying relevance to object geometry. To address this issue, we propose AVA-3D, an Attribute-based Visual Arbitration framework. AVA-3D introduces a Mixture of Arbitration (MoA) module to reorganize view-indexed features into complementary latent visual attributes through dual-axis competitive arbitration, together with a Conditional Attribute Fusion (CAF) module that adaptively routes and transforms these attributes according to 3D geometry. By enabling attribute-level organization across views and geometry-conditioned fusion across modalities, AVA-3D learns more discriminative 3D representations. Extensive experiments on Objaverse, ModelNet40, and ScanObjectNN demonstrate consistent improvements across downstream tasks.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
3D representation learning, multimodal fusion, multi-view learning, point cloud

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_DTTQOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
2(+1)
Read review
Paper summary

This paper addresses multi‑view visual heterogeneity and modality gap in multimodal 3D object representation learning. The proposed AVA‑3D framework consists of two core modules: Mixture of Arbitration (MoA) which transforms view‑level multi‑view features into a set of latent complementary visual attributes via dual‑axis competitive arbitration, and Conditional Attribute Fusion (CAF), which adaptively routes and applies attribute‑specific residual updates conditioned on point‑cloud geometry to enhance 3D representations. The model is trained with multimodal contrastive losses. Evaluations are conducted on Objaverse‑LVIS, ModelNet40 and ScanObjectNN under different pre‑training data scales, including ablation studies, slot / view hyper‑parameter analysis, attribute correlation and intervention analysis, plus cross‑modal retrieval experiments.

Critical strengths
  1. Well‑defined problem statement for multi‑view heterogeneous fusion. The paper clearly identifies the limitation of existing global / view‑level multi‑view aggregation, and proposes to shift to attribute‑level representation learning. The motivation is reasonable for multimodal 3D pre‑training research.

  2. Diverse set of empirical analyses beyond main benchmark numbers. Besides zero‑shot classification, the paper provides slot‑attribute correlation analysis and controlled attribute intervention experiments to interpret what MoA slots learn, which brings additional insight for understanding the module behavior.

Critical weaknesses
  1. Crucial overview Figure 2 has severe rendering defects that harm paper readability and reproducibility assessment. In the provided manuscript PDF, Figure 2 (the full pipeline overview) suffers from missing text labels and indistinguishable grey‑purple graphic elements. This is the central diagram explaining MoA‑CAF workflow. Without legible figure, reviewers cannot reliably verify whether the architectural description matches the diagram, which is a major presentation flaw for conference submission.

  2. Limited practical grounding of latent visual attributes. MoA produces purely latent slots without explicit semantic supervision. Although correlation and intervention tests are provided, there is no clear mechanism to guarantee slots consistently correspond to meaningful visual attributes across different datasets / object distributions. The claimed “attribute‑level” abstraction remains largely post‑hoc interpreted rather than guaranteed by model design. In addition, overall experimental workload is moderate, and gains on several real‑world benchmarks are only marginal compared to strong existing baselines.

Questions for authors
  1. Please provide a properly‑rendered, complete version of Figure 2 with all text labels visible and clear visual distinction between grey and purple graphical components. Clarify each intermediate tensor flow shown in this pipeline diagram.

  2. Are the learned MoA slot‑to‑visual‑attribute associations transferable when evaluated on completely unseen object categories? Can you provide quantitative metrics measuring slot consistency across different object sets, instead of only correlation / intervention examples on Objaverse?

  3. In Table 1, under the full Ensembled pre‑training setting, AVA‑3D only brings small gains over strong baselines on ModelNet40. Please analyze why the improvement is much weaker on this dataset compared to Objaverse‑LVIS.

4 comments
AuthorsOpenReview ↗

Reply to Reviewer DTTQ (Part1)

Thank you for your thoughtful and constructive review. We appreciate your recognition of our motivation for attribute-level multi-view fusion and the insights provided by our correlation and intervention analyses. In response, we have revised Figure 2 to improve its visual clarity and readability and clarified the interpretation and limitations of implicitly learned attributes. We have also added a quantitative evaluation of slot–attribute consistency across seen and held-out object sets and analyzed the smaller improvements on ModelNet40 relative to Objaverse-LVIS. We address each concern in detail below.

W1, Q1 (Figure 2 Readability)

“In the provided manuscript PDF, Figure 2 (the full pipeline overview) suffers from missing text labels and indistinguishable grey-purple graphic elements.”

Thank you for bringing this to our attention. We carefully inspected the original PDF but could not reproduce the reported rendering issue. Nevertheless, we have revised Figure 2 by enlarging the text labels, strengthening the color contrast, and clarifying the intermediate tensor flows to improve overall readability. The updated figure is included in the revised manuscript.

W2 (Practical Grounding of Latent Visual Attributes)

“MoA produces purely latent slots without explicit semantic supervision. […] there is no clear mechanism to guarantee slots consistently correspond to meaningful visual attributes across different datasets / object distributions.”

Thank you for raising this concern. We clarify that our attributes are data-driven latent representations, not explicitly defined semantic concepts. MoA organizes complementary evidence from heterogeneous views, and CAF integrates it with point-cloud geometry under multimodal contrastive supervision. Thus, “attribute-level” describes the granularity of evidence organization and fusion; it does not imply a guaranteed one-to-one correspondence with a predefined semantic vocabulary.

We also respectfully disagree that the improvements are uniformly marginal. On Objaverse-LVIS (1,156 categories), AVA-3D achieves 55.8% zero-shot Top-1 accuracy, surpassing MRD-L (53.6%) while using a much smaller reported 3D backbone (32.5M vs. 306.7M parameters). On real-scanned ScanObjectNN, it improves over DuoDuoCLIP from 60.7% to 65.2%. On OmniObject3D (190 real-scanned categories), class-level text-to-shape R@1 rises from 23.96% to 39.12% relative to DuoDuoCLIP. These results establish practical benefits without implying fixed semantic identities for individual slots.

Implicit attribute learning also offers flexibility: rather than constraining evidence to a fixed attribute vocabulary, MoA can capture different combinations of cues across objects and viewpoints. The low mean pairwise cosine similarity (0.19) between slot profiles supports differentiated preferences, although it does not guarantee explicit semantic grounding. As discussed in limitation section, we will explore semantic supervision or lightweight attribute constraints to improve interpretability and consistency while retaining this flexibility.

AuthorsOpenReview ↗

Reply to Reviewer DTTQ (Part2)

Q2 (Transferability of Slot–Attribute Associations)

“Are the learned MoA slot-to-visual-attribute associations transferable when evaluated on completely unseen object categories? Can you provide quantitative metrics measuring slot consistency across different object sets?”

Thank you for suggesting this evaluation. Using the model pretrained on Ensembled (No LVIS), we evaluate slot–attribute consistency between seen categories and completely unseen Objaverse-LVIS categories. For each slot, we construct an attribute profile by averaging the z-scored visual-attribute values of its selected views within each object set, then measure the similarity of these profiles across sets. This establishes cross-set consistency on held-out objects; a claim of completely unseen-category transfer additionally requires a category-disjoint split.

As shown in Table G, the same-slot cosine similarity reaches 0.96, substantially higher than the 0.20 similarity between different slots. Furthermore, the 0.78 Top-5 attribute overlap indicates that the dominant attribute preferences are largely preserved across the evaluated object sets. These results provide quantitative evidence of consistent yet differentiated attribute preferences on held-out objects.

MetricResultComparison
Same-slot cosine similarity0.96Same-slot attribute profiles across sets
Cross-slot cosine similarity0.20Different-slot attribute profiles across sets
Top-5 attribute overlap0.78Fraction of shared top-five attributes for the same slot across sets

Table G. Cross-set consistency of selected-view attribute profiles.

Q3 (Gains on ModelNet40)

“Please analyze why the improvement is much weaker on this dataset compared to Objaverse-LVIS.”

Thank you for raising this question. ModelNet40 views rendered from largely textureless CAD-derived point clouds provide limited appearance information beyond geometry, reducing the complementary evidence available to MoA and CAF. Recognition therefore relies more heavily on the point-cloud branch. In contrast, Objaverse-LVIS offers richer, more heterogeneous views and a substantially more challenging long-tailed evaluation set (46,832 objects across 1,156 categories, versus 2,468 test objects across 40 categories in ModelNet40).

Consistent with this difference, under Ensembled (Full) pretraining, AVA-3D improves over MixCon3D by 0.9 points on ModelNet40 (87.7% vs. 86.8%), but by 3.3 points on Objaverse-LVIS (55.8% vs. 52.5%), where complementary visual evidence is more informative.

Thank you for the reviewers’ careful feedback. Most of my concerns have been addressed, and I am willing to raise my score to 3.

Good luck!

AuthorsOpenReview ↗

Thank You for Your Feedback

We are glad that our response has addressed your concerns. We sincerely appreciate your thoughtful review and positive feedback.

Reviewer_usRMOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how to incorporate multi-view image information into point cloud representations under tri-modal contrastive pre-training. Building on MixCon3D, the authors propose AVA-3D, which consists of two modules: a Mixture of Arbitration (MoA) that uses learnable slots with dual-axis normalization to reorganize view embeddings into latent "attribute" tokens, and a Conditional Attribute Fusion (CAF) that routes these tokens based on the point cloud embedding and adds the resulting residual corrections back to it. Experiments mainly evaluate zero-shot classification on Objaverse-LVIS, ModelNet40, and ScanObjectNN, supplemented by text-to-shape retrieval, ablation studies, and an analysis of the learned slots.

Critical strengths
  1. The design is lightweight and easy to adopt, yielding consistent gains across experiments, especially in low data regimes. Simply plugging these two modules into a MixCon3D style pipeline using the exact same backbone and the public OpenShape dataset leads to clear and significant improvements.

  2. The diagnostic findings regarding view heterogeneity are highly valuable. Figure 4 shows that single-view accuracy varies widely, ranging from 32.7% to 52.3%. Interestingly, simple average pooling over all 12 views yields only 44.8%, which actually underperforms the best single view, whereas the learnable slot-based aggregation pushes the accuracy to 54.7%. This provides concrete evidence that naive multi-view pooling—the default practice in this line of work—inevitably discards useful information.

Critical weaknesses
  1. The claims overreach the evidence. The abstract states "consistent improvements across downstream tasks," yet the quantitative support amounts to zero-shot classification and a single Text2Shape retrieval table covering only two categories (chair and table).

  2. There is a tension between the stated motivation and the actual method. The introduction argues that aggregating views into a unified representation "may not fully preserve the diversity of visual evidence across viewpoints." However, per Equations (7) and (12), AVA-3D itself performs view aggregation. Presumably what the authors intend is: "learnable, conditioned weighted aggregation is superior to mean pooling." But once framed this way, the work sits squarely in the same family as existing learned view-weighting and view-selection methods (e.g., GVCNN, MV-CLIP, KR-MAE, OpenView). The paper never compares against a simple learnable view-weighting baseline within its own pipeline, making it impossible to determine whether the gains stem from the proposed arbitration mechanism or simply from the well-known fact that weighted aggregation outperforms uniform pooling.

Notes: GVCNN: Group-View Convolutional Neural Networks for 3D Shape Recognition. MV-CLIP: Multi-View CLIP for Zero-shot 3D Shape Recognition. KR-MAE:Reliable-view 2D-3D Key-part Aligned Transformer with Reinforced Masking for 3D Point Cloud Understanding. OpenView:Cross-Modal 3D Representation with Multi-View Images and Point Clouds.

Questions for authors
  1. In Table 2, for the "Baseline + CAF" variant, what tokens does CAF actually receive when MoA is removed? If it only takes a single global visual token, then α\alpha is trivially 1 and the routing mechanism does nothing. Given this, how can the performance gain over the baseline be attributed to "geometry-conditioned routing" rather than simply the geometry-conditioned residual adapters?

  2. How exactly are the attribute metrics in Figure 6 defined and computed?

  3. In Figure 6, the absolute correlation coefficients never exceed 0.27, which is kind of weak. More importantly, every single slot is negatively correlated with "View distinctiveness" and positively correlated with "Semantic clarity." Is this really expected if the slots are supposed to capture distinct and complementary visual attributes?

  4. In Appendix Table 4, MXM-CLR actually outperforms AVA-3D on text-to-shape retrieval (RR@1 16.83 vs. 15.58). This directly contradicts the blanket SOTA claims made in the introduction and abstract.

  5. The color coding in the framework diagram (Figure 2) is confusing. For instance, V2V_2 and s2s_2 are both colored orange, which could mislead readers into thinking slot s2s_2 is tied specifically to view V2V_2 or "Object 2 Table". Could you revise the figure to use a clearer color scheme that visually decouples the view indices from the learnable slot indices?

6 comments
AuthorsOpenReview ↗

Reply to Reviewer usRM (Part1)

Thank you for your detailed and constructive review. We appreciate your recognition of our lightweight and easy-to-adopt design, its effectiveness in low-data settings, and our findings on the limitations of naive multi-view pooling. In response, we have extended retrieval evaluation to OmniObject3D and compared MoA with learnable view weighting and alternative aggregation mechanisms. We also clarify the CAF ablation, provide detailed definitions of the attribute metrics and further analysis of slot preferences, explain the differences in Text2Shape training settings, and revise the color scheme of Figure 2. We address each concern in detail below.

W1, Q4 (Scope of Claims and Retrieval Evaluation)

“The claims overreach the evidence. The abstract states ‘consistent improvements across downstream tasks,’ yet the quantitative support amounts to zero-shot classification and a single Text2Shape retrieval table covering only two categories (chair and table).”

“In Appendix Table 4, MXM-CLR actually outperforms AVA-3D on text-to-shape retrieval (RR@1 16.83 vs. 15.58). This directly contradicts the blanket SOTA claims made in the introduction and abstract.”

Thank you for raising these concerns. To broaden our retrieval evaluation beyond the two categories in Text2Shape, we have conducted additional text-to-shape retrieval experiments on OmniObject3D, which contains approximately 6,000 real-scanned objects spanning 190 everyday categories.

As shown in Table E, AVA-3D outperforms DuoDuoCLIP across all eight retrieval metrics, improving instance-level R@1 from 1.38% to 1.81% and class-level R@1 from 23.96% to 39.12%.

MethodInst. R@1Inst. R@3Inst. R@5Inst. R@10Class R@1Class R@3Class R@5Class R@10
DuoDuoCLIP1.383.796.3211.2623.9635.9345.2857.41
AVA-3D (Ours)1.815.498.9514.8439.1256.4863.1371.43

Table E. Text-to-shape retrieval results (%) on OmniObject3D. Inst. and Class denote instance-level and class-level retrieval, respectively; higher is better.

Regarding Text2Shape, we acknowledge that MXM-CLR achieves a higher RR@1 score than AVA-3D (16.83% vs. 15.58%). However, MXM-CLR is specifically trained on Text2Shape, whereas AVA-3D follows a general-purpose pretraining setting. Among methods with comparable pretraining settings in Appendix Table 4, AVA-3D achieves the highest retrieval accuracy. We will clarify this distinction and refine the SOTA claims in the abstract and introduction to avoid overgeneralization.

Together, these additional results provide stronger evidence for AVA-3D's effectiveness on both zero-shot classification and text-to-shape retrieval, extending retrieval evaluation to a substantially broader range of categories and real-scanned objects. We will further expand the OmniObject3D evaluation to include more comprehensive comparisons with additional baselines in the final version.

Q1 (CAF without MoA)

“Given this, how can the performance gain over the baseline be attributed to ‘geometry-conditioned routing’ rather than simply the geometry-conditioned residual adapters?”

Thank you for pointing out this ambiguity. In the “Baseline + CAF” variant, CAF receives a single global visual token when MoA is removed. Consequently, the routing weight becomes 1, and no attribute-level selection occurs. However, the geometry-conditioned residual adapter remains effective: it uses the global visual feature, point-cloud representation, and their element-wise interactions and differences to generate a geometry-dependent feature update. Thus, the performance gain in this variant should be attributed to geometry-conditioned residual adaptation rather than attribute routing. In the full model, MoA provides multiple distinct attribute representations, enabling CAF to perform both geometry-conditioned attribute routing and residual adaptation. We will clarify this distinction and correct the attribution in the ablation discussion.

AuthorsOpenReview ↗

Reply to Reviewer usRM (Part2)

W2 (Motivation and View Weighting)

“The paper never compares against a simple learnable view-weighting baseline within its own pipeline, making it impossible to determine whether the gains stem from the proposed arbitration mechanism or simply from the well-known fact that weighted aggregation outperforms uniform pooling.”

Thank you for highlighting this concern. We would like to clarify that our motivation is not to avoid view aggregation itself, but to preserve diverse visual evidence through multiple latent attribute representations before geometry-conditioned fusion, rather than collapsing all views into a single global embedding. To validate this distinction, we have added a learnable view-weighting baseline within the same pipeline. Following Reviewer 1's suggestion, we also evaluated Perceiver-style pooling, slot competition, and dual-softmax under the same view setting.

VariantTop-1 ↑Top-3 ↑Top-5 ↑
Mean pooling25.6442.1350.02
Perceiver-style pooling32.1150.4857.79
Slot competition28.8246.4553.78
Dual-softmax28.1845.0052.51
Learnable weights27.8945.9453.87
MoA (Ours)34.9053.4061.00

Table F. ShapeNet-only zero-shot classification accuracy (%) on Objaverse-LVIS.

Under ShapeNet-only training, learnable view weighting improves Top-1 accuracy from 25.64% to 27.89%, confirming the benefit of adaptive weighting over uniform pooling. MoA further achieves 34.90%, outperforming learnable weighting by 7.01 percentage points and the strongest alternative by 2.79 percentage points, while also achieving the highest Top-3 and Top-5 accuracy.

These results demonstrate that MoA's gains extend beyond simple learnable view weighting. By jointly modeling view relevance and competition among attribute slots, MoA constructs multiple attribute representations through competitive allocation and residual updates, which CAF can subsequently select and adapt based on point-cloud geometry. This enables geometry-conditioned fusion to exploit complementary visual evidence rather than relying on a single aggregated visual representation.

Q2 (Attribute Metrics and Computation)

“How exactly are the attribute metrics in Figure 6 defined and computed?”

Thank you for requesting these details. For the 12-view analysis in Figure 6 (separate from the four-view default inference setting), we compute, for each object, slot, and attribute, the Pearson correlation between the pre-softmax routing logits and the corresponding view-level attribute scores, then average the resulting correlations across objects. These object-averaged correlations characterize each slot's view-selection preferences. We consider four low-level attributes (centeredness, elongation, bounding-box area, and horizontal symmetry) and seven high-level attributes (color salience, texture richness, planarity, shape complexity, semantic symmetry, view distinctiveness, and semantic clarity). Detailed attribute definitions and computation procedures are provided in Appendix A.2 of the revised manuscript.

AuthorsOpenReview ↗

Reply to Reviewer usRM (Part3)

Q3 (Correlation Strength and Slot Diversity)

“In Figure 6, the absolute correlation coefficients never exceed 0.27, which is kind of weak. More importantly, every single slot is negatively correlated with ‘View distinctiveness’ and positively correlated with ‘Semantic clarity.’”

Thank you for raising this concern. Figure 6 measures the correlations between routing preferences and individual predefined attributes, whereas MoA learns combinations of multiple visual and semantic cues that cannot be fully characterized by any single descriptor. For comparison, the maximum absolute correlation under standard Slot Attention is 0.13, compared with 0.27 for MoA.

The shared negative correlations with View distinctiveness indicate a preference for representative views over feature-space outliers, as this metric measures deviation from the object's mean CLIP view feature. Similarly, the shared positive correlations with Semantic clarity indicate a preference for views that align well with the object's textual descriptions. However, these common tendencies do not necessarily imply identical attribute preferences across slots.

To further examine slot diversity, we analyze selected-view attribute profiles on Objaverse-LVIS by averaging the z-scored attribute values of the views selected by each slot. The mean pairwise cosine similarity between different slots is 0.19, suggesting that slots emphasize distinct combinations of visual attributes despite sharing general view-selection preferences.

Q5 (Figure 2 Color Coding)

“Could you revise the figure to use a clearer color scheme that visually decouples the view indices from the learnable slot indices?”

Thank you for pointing this out. We agree that the original color scheme could misleadingly suggest a correspondence between view indices and learnable slot indices. We have revised Figure 2 to use distinct and consistent color conventions for views, learnable slots, and object examples, and uploaded the updated figure.

Reviewer_ZvVHOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

AVA-3D extends OpenShape-style tri-modal contrastive pre-training (PointBERT point encoder, frozen OpenCLIP ViT-bigG-14 image and text encoders, OpenShape data and captions) with two modules that operate on per-view CLIP image embeddings. MoA lets K = 8 learnable slots attend to M = 4 view embeddings, multiplying a softmax over views by a softmax over slots and renormalizing over views, to form "attribute" tokens. CAF scores each attribute against the point embedding using [p; a; p*a; |p-a|], softmax-normalizes the scores, and adds a weighted sum of per-attribute low-rank residuals to p. Contrastive losses supervise both p and the fused embedding h, and h is used for zero-shot classification.

Critical strengths

Sec. 4.4 contains a concrete observation worth keeping: single rendered views range from 32.7 to 52.3 Top-1 on Objaverse-LVIS, and mean pooling over 12 views (44.8) is below the best single view, which is a clear argument against naive multi-view averaging. Training is cheap (122 A800 GPU-hours), and the slot-count ablation is clean.

Critical weaknesses

Table 1 does not say what AVA-3D sees at test time, and the answer determines how it should be read. The embedding h is a function of rendered views of the test object (Sec. 3.4; Sec. 4.3 varies the number of views at inference from 1 to 12). OpenShape, ULIP, TAMM, Uni3D, MRD, and ViT-Lens classify from the point cloud alone, and DuoDuoCLIP from images alone, but the table has no modality column, and the paper never says how views are obtained for ModelNet40 and ScanObjectNN, which have no textured meshes.

The two modules are close to standard components that are neither cited nor compared. Normalizing attention over slots and then renormalizing over inputs is the competition mechanism of Slot Attention (Locatello et al., NeurIPS 2020); multiplying it by a softmax over the other axis is the dual-softmax used for matching in LoFTR (Sun et al., CVPR 2021); learned latent queries over input tokens are the Perceiver (Jaegle et al., ICML 2021). CAF is a softmax-gated mixture of low-rank adapters on the familiar [p; a; p*a; |p-a|] matching features. MoA is compared with no module, with mean pooling (Sec. 4.4), and with K = 1, whose row in Table 3 is identical to "Baseline + CAF" in Table 2 (28.1 / 45.6 / 52.8); it is never compared with a single-axis softmax, slot attention, or Perceiver pooling of the same four embeddings, so the benefit of the dual-axis part is not shown.

Questions for authors
  1. For every row of Table 1, AVA-3D included, which modalities and how many views are used at test time? How are views produced for ModelNet40 and ScanObjectNN?
  2. Zero-shot accuracy of p alone, and of an image-only classifier with the same frozen encoder and views (mean embedding and MoA output), in all three training settings.
  3. MoA against a single-axis softmax, slot attention, and Perceiver-style cross-attention with the same K and views.
  4. Per-object inference cost including the image encoder, compared with Uni3D-L and DuoDuoCLIP.
5 comments
AuthorsOpenReview ↗

Reply to Reviewer ZvVH (Part1)

Thank you for your detailed and constructive review. We appreciate your recognition of our findings on view heterogeneity, training efficiency, and slot-count ablation. In response to your concerns, we clarify the test-time inputs, provide additional comparisons with alternative aggregation mechanisms, and further analyze the category-discriminative information in routing patterns. We also report point-only, image-only, and fused performance across all three training settings, along with inference costs including image encoding. We address each point in detail below.

W1, Q1 (Test-Time Inputs and View Counts)

“For every row of Table 1, AVA-3D included, which modalities and how many views are used at test time? How are views produced for ModelNet40 and ScanObjectNN?”

Thank you for identifying this ambiguity. AVA-3D uses point clouds and four rendered image views at inference. For ModelNet40 and ScanObjectNN, the views are rendered from the corresponding point clouds. OpenShape, ULIP, TAMM, Uni3D, MRD, and ViT-Lens use point clouds alone, without test-time images. PointCLIP, PointCLIP v2, and CLIP2Point use 6, 10, and 6 projected depth views, respectively, following their official zero-shot settings. DuoDuoCLIP uses five rendered views, while MixCon3D combines point clouds with four rendered views on Objaverse-LVIS. We have updated Table 1 to specify each method's test-time modalities and view counts. We will also release our rendering code and generated images to facilitate reproducibility.

Q2 (Unimodal and Fused Performance)

“Zero-shot accuracy of p alone, and of an image-only classifier with the same frozen encoder and views (mean embedding and MoA output), in all three training settings.”

Thank you for suggesting this comparison. We evaluated the original OpenShape point-cloud baseline (p-base) and four representation variants on Objaverse-LVIS across all three pretraining settings, using the same frozen image encoder and rendered views for all image-based variants.

Training settingp-base (OpenShape)p aloneImage mean embeddingImage MoA outputFull AVA-3D
Ensembled (Full)46.80 / 69.10 / 77.0052.10 / 73.81 / 80.3144.06 / 64.68 / 71.5953.09 / 73.72 / 79.9755.75 / 77.23 / 83.42
Ensembled (No LVIS)39.10 / 60.80 / 68.9044.00 / 65.21 / 72.7344.06 / 64.68 / 71.5952.98 / 73.61 / 79.9453.38 / 74.39 / 80.87
ShapeNet-only10.80 / 20.20 / 25.0011.28 / 20.26 / 24.5244.06 / 64.68 / 71.5940.25 / 59.88 / 67.1237.99 / 57.46 / 64.93

Table C. Zero-shot Top-1 / Top-3 / Top-5 accuracy (%) on Objaverse-LVIS. p-base denotes the original OpenShape point-cloud baseline, while p alone denotes the point-cloud representation learned by AVA-3D. The mean image embedding is training-independent and therefore yields identical results across settings.

First, AVA-3D improves the point-cloud representation even without test-time image inputs. Compared with OpenShape, which shares the same point-cloud backbone and comparable pretraining settings, the point-cloud branch of AVA-3D achieves Top-1 improvements of 5.30, 4.90, and 0.48 percentage points under Ensembled (Full), Ensembled (No LVIS), and ShapeNet-only pretraining, respectively. These gains suggest that joint optimization with attribute-aware fusion can also benefit the learned geometric representation.

Second, AVA-3D integrates visual and geometric evidence across different pretraining settings. Under ShapeNet-only pretraining, despite limited training data, MoA learns informative visual representations, achieving 40.25% Top-1 accuracy versus only 11.28% for the point-cloud branch. Despite this substantial modality imbalance, CAF retains 37.99% accuracy after fusion, only 2.26 percentage points below image-only MoA. These results demonstrate MoA’s effectiveness under limited supervision and CAF’s robustness to imbalanced modality representations. With broader pretraining coverage, MoA improves image-only accuracy from 44.06% to 53.09% using the same frozen encoder and views. Under Ensembled (Full), the fused representation reaches 55.75%, outperforming both individual branches. A similar pattern holds under Ensembled (No LVIS), where image-only MoA achieves 52.98% and the fused representation reaches 53.38%.

These results support complementary roles for our two designs: MoA organizes visual evidence through attribute-level aggregation, while CAF adaptively integrates these attributes with point-cloud geometry. Fusion improves upon both individual branches under broader pretraining, while retaining much of the visual branch's performance in the ShapeNet-only setting.

AuthorsOpenReview ↗

Reply to Reviewer ZvVH (Part2)

W2, Q3 (Comparison with Alternative Aggregation Designs)

“MoA against a single-axis softmax, slot attention, and Perceiver-style cross-attention with the same K and views.”

Thank you for highlighting these connections. We would like to clarify that our contribution lies in the integrated attribute-organization design for geometry-conditioned fusion, rather than in individual attention operations. Specifically, MoA models the many-to-many relationships between partial views and latent attributes: each attribute can aggregate evidence from multiple views, while each view can contribute to multiple attributes. Learned queries identify relevant views, while dual-axis weighting captures both view relevance and competition among attribute slots. Residual updates then construct the attribute representations. Although MoA shares certain underlying operations with Perceiver cross-attention, Slot Attention, and dual-softmax, it integrates view relevance and attribute competition for attribute construction, rather than relying solely on independent query-wise pooling, slot-based competition, or pairwise matching scores.

Under the same view setting, MoA outperforms the strongest evaluated alternative on Objaverse-LVIS by 2.79, 2.92, and 3.21 percentage points in Top-1, Top-3, and Top-5 accuracy, respectively (Table A), demonstrating the effectiveness of the integrated design.

VariantTop-1 ↑Top-3 ↑Top-5 ↑
Mean pooling25.6442.1350.02
Perceiver-style pooling32.1150.4857.79
Slot competition28.8246.4553.78
Dual-softmax28.1845.0052.51
Learnable weights27.8945.9453.87
MoA (Ours)34.9053.4061.00

Table A. ShapeNet-only zero-shot classification accuracy (%) on Objaverse-LVIS.

We further investigate whether the learned routing patterns capture category-related information on Objaverse-LVIS. Each object is represented by its normalized slot–view routing weights, which are evaluated using three metrics:

  • Inter/Intra Ratio: (Dinter+ϵ)/(Dintra+ϵ)(D_{\mathrm{inter}}+\epsilon)/(D_{\mathrm{intra}}+\epsilon), comparing mean inter-class and intra-class cosine distances. Values above 1 indicate greater separation between categories.

  • Fisher Ratio: B/(W+ϵ)B/(W+\epsilon), comparing between-class and within-class scatter. Higher values indicate better category separation. Both ratios use category-balanced weighting with ϵ=10−8\epsilon=10^{-8}.

  • Hit@K: The percentage of queries with at least one same-category object among their K nearest neighbors, excluding self-matches. Hit@1 is equivalent to 1-NN classification accuracy.

The results are presented in Table B.

MethodInter/Intra Ratio ↑Fisher Ratio ↑Hit@1 ↑Hit@5 ↑Hit@10 ↑
Mean pooling1.000.000.15%0.73%1.45%
Perceiver-style pooling1.090.147.10%13.27%17.56%
Slot competition1.100.130.23%0.87%1.64%
Dual-softmax1.060.110.88%3.15%5.56%
MoA (Ours)1.210.2718.11%30.18%36.16%

Table B. Category-related structure in routing patterns.

Compared with the strongest alternative for each metric, MoA increases the Inter/Intra Ratio by 0.11 to 1.21 and the Fisher Ratio by 0.13 to 0.27. It also improves Hit@1/5/10 by 11.01 / 16.91 / 18.60 percentage points, reaching 18.11% / 30.18% / 36.16%, respectively. These results indicate that MoA learns more category-discriminative routing patterns, supporting its effectiveness in organizing visual evidence for subsequent attribute extraction and geometry-conditioned fusion. Together with the classification improvements in Table A, these findings demonstrate the advantages of the integrated MoA design over the evaluated alternative aggregation mechanisms. We will also add explicit citations to Perceiver, Slot Attention, and LoFTR and expand the discussion of their connections and differences in the supplementary material.

AuthorsOpenReview ↗

Reply to Reviewer ZvVH (Part3)

Q4 (Per-Object Inference Cost)

“Per-object inference cost including the image encoder, compared with Uni3D-L and DuoDuoCLIP.”

Thank you for suggesting this comparison. We benchmarked all three methods on the same hardware, including image encoding where applicable. The results are presented in Table D.

MethodTime / object ↓FLOPs ↓Peak memory ↓Params ↓
AVA-3D (Ours)17.06 ms43.70 GFLOPs0.59 GiB139.08M
DuoDuoCLIP7.20 ms44.79 GFLOPs0.59 GiB151.28M
Uni3D-L34.38 ms310.56 GFLOPs1.34 GiB307.35M

Table D. Per-object inference costs measured on the same hardware, including image encoding for image-based methods.

Compared with Uni3D-L, AVA-3D achieves approximately 2× faster inference (17.06 vs. 34.38 ms/object), with substantially lower FLOPs and peak memory. Meanwhile, it achieves higher Top-1 accuracy on both Objaverse-LVIS (55.8% vs. 53.1%) and ScanObjectNN (65.2% vs. 58.2%).

Compared with DuoDuoCLIP, AVA-3D incurs higher inference latency (17.06 vs. 7.20 ms/object) while exhibiting comparable FLOPs and peak memory, with both methods using the same image encoder. However, AVA-3D achieves higher Top-1 accuracy on Objaverse-LVIS (55.8% vs. 54.1%) and ScanObjectNN (65.2% vs. 60.7%). Importantly, unlike DuoDuoCLIP, which relies entirely on multi-view images and requires point-cloud inputs to be converted into rendered or projected views, AVA-3D directly encodes the original point cloud alongside visual evidence, preserving access to explicit 3D geometric information. These results demonstrate that AVA-3D achieves a favorable balance between inference efficiency and recognition performance while supporting both geometric and visual inputs.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response — Thanks to All Reviewers for Their Constructive Feedback

We sincerely thank the Area Chair and all reviewers for their careful evaluation and constructive feedback. We are encouraged by your recognition of AVA-3D's motivation, lightweight design, and effectiveness in addressing heterogeneous multi-view visual information.

Reviewer ZvVH highlighted the value of our findings on view heterogeneity, particularly the limitations of naive multi-view averaging, and appreciated the low training cost and clear ablation studies. Reviewer usRM recognized the lightweight and easy-to-adopt design, the improvements under limited pretraining data, and the diagnostic evidence supporting attribute-level aggregation. Reviewer DTTQ appreciated the motivation for addressing heterogeneous multi-view fusion and the attribute correlation and intervention analyses that help explain MoA's behavior.

During the rebuttal period, we conducted additional experiments and revised the manuscript to address the reviewers' concerns:

  1. Evaluation protocols and modality contributions (Reviewer ZvVH). We clarified the test-time modalities, view counts, and rendering protocols of the compared methods. We also evaluated point-cloud-only, image-only, and fused representations across all three pretraining settings to examine the contributions of MoA and CAF.

  2. Alternative aggregation mechanisms (Reviewers ZvVH and usRM). We compared MoA with learnable view weighting, Perceiver-style pooling, Slot Attention, and dual-softmax under matched settings. We further analyzed the category-discriminative information in their routing patterns using Inter/Intra Ratio, Fisher Ratio, and nearest-neighbor retrieval metrics.

  3. CAF mechanism and attribute interpretation (Reviewers usRM and DTTQ). We clarified the role of geometry-conditioned residual adaptation when CAF operates without MoA. We also provided detailed attribute definitions, analyzed shared and slot-specific selection preferences, and evaluated slot diversity and cross-set consistency while acknowledging the limitations of implicit semantic grounding.

  4. Broader retrieval evaluation (Reviewer usRM). We extended text-to-shape retrieval experiments to OmniObject3D, covering 190 real-scanned object categories. We also clarified the differences in Text2Shape pretraining settings and refined our performance claims to better reflect the experimental evidence.

  5. Inference efficiency (Reviewer ZvVH). We compared AVA-3D with Uni3D-L and DuoDuoCLIP in terms of inference latency, FLOPs, peak memory, and parameter counts under the same hardware settings.

  6. Dataset-specific analysis and framework presentation (Reviewers usRM and DTTQ). We explained the smaller improvements on ModelNet40 compared with Objaverse-LVIS, revised Figure 2 to improve visual clarity and tensor-flow presentation, and expanded the discussion of the framework's limitations.

These results and clarifications have been incorporated into the revised manuscript, including Figure 2, Table 1, and the Attribute Analysis section, with further implementation details and experiments provided in the supplementary More Experimental Details and Additional Experimental Results sections. The corresponding textual revisions are highlighted in red.

We are particularly grateful for the positive feedback received during the discussion. Reviewer usRM acknowledged that our responses had addressed most of their concerns and increased their rating to 3 (Weak Accept). Similarly, Reviewer DTTQ indicated that most concerns had been resolved and expressed willingness to raise their rating to 3 (Weak Accept), as reflected in the updated review. We sincerely appreciate both reviewers' continued engagement and positive reassessment of our work.

We believe these additional experiments and clarifications have strengthened the empirical support for AVA-3D and provided a clearer understanding of its contributions and limitations. We will incorporate the remaining feedback into the final manuscript to further improve its clarity and completeness.

Once again, we sincerely thank all reviewers and the Area Chair for their valuable feedback, time, and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.