TETRIS: Self-Supervised Task Representation Learning for Point In-Context Learning

Point In-Context Learning (Point ICL) aims to infer a transformation from a support pair and apply it to a query point cloud without task-specific parameter updates. However, existing protocols train and evaluate models on the same predefined task families, allowing a task-recognition shortcut: models may identify and execute a familiar task rather than infer the transformation from context. Such evaluations leave transfer to unseen transformation processes unresolved. To assess such transfer, we introduce PIC-Bench, a benchmark of five structured point cloud restoration tasks modeling corruptions arising in 3D acquisition and data processing, spanning both deterministic and ambiguous transformations. PIC-Bench reveals a substantial generalization gap in existing Point ICL methods, suggesting that success on familiar tasks does not readily transfer to unseen transformation processes. To address this limitation, we propose Task rEpresenTation leaRnIng through Self-supervision (TETRIS), a self-supervised task representation learning framework requiring no training pairs from downstream tasks. TETRIS uses unlabeled clean point clouds as targets and generates corresponding sources through online compositions of parameterized atomic point-set operators. Applying the same composition across different objects encourages the model to represent transformations independently of object geometry. On PIC-Bench, TETRIS achieves the best performance across all five tasks and reduces the overall Chamfer distance from $31.6$ to $8.7$ compared with the same backbone trained under the original supervised protocol. When fine-tuned on downstream ShapeNet-In-Context tasks, TETRIS pre-training further improves data efficiency, reducing the Chamfer distance from $9.8$ to $4.9$ with only $1\%$ of the labeled data.