3D Open-Modality Learning with Incomplete Point-Cloud Supervision

Multimodal pretraining enriches point-cloud representations with image and text supervision, but commonly relies on paired point-cloud data. When point clouds are unavailable for some training objects, their image–text information cannot directly supervise the point encoder used at deployment. We study 3D Open-Modality learning, where point clouds cover only a subset of training objects and inference uses real point clouds alone. We propose Two-stage Point Supervision Transfer (TPST), which separates privileged supervision transfer from calibration of the final point readout. First, Text-Conditioned Point Replay (TCPR) constructs image-conditioned proxy tokens, modulates them with same-object text, and calibrates them against available real-point representations. These proxies carry supervision from objects without point clouds through the retained point encoder, while real points provide geometric grounding. Second, Geometry-Preserved Shared Readout (GPSR) adds a shared residual readout with a bounded scalar gate while freezing the Stage-1 point base and proxy module. The complete point-only deployment keeps two frozen readouts: an independently trained geometry branch (B0) and the calibrated branch. A frozen source-only selector chooses the branch on Top-1 disagreement, and the calibrated scores may rerank only the B0 Top-5 support, so the final Top-5 set is unchanged. At 60% point-cloud availability, TPST improves zero-shot Top-1 accuracy over matched B0 by 0.65 pp on ModelNet40, 2.07 pp on ScanObjectNN OBJ_ONLY, and 0.87 pp on PB_T50_RS. We further evaluate representation quality through frozen-encoder few-shot linear probing and controlled ablations. Code, configurations, and evaluation scripts will be released with the paper.