DRIP-MPU: Distilled Context Retrieval for Few-Shot Tabular MPU Classification

Few-shot tabular multi-positive and unlabeled (MPU) classification requires distinguishing multiple positive classes and an unknown negative class using only a few labeled positives and an unlabeled pool. Scarce supervision limits task-specific PU/MPU learning, while the absence of labeled negatives prevents the direct use of tabular foundation models that require every class to be represented in context. We propose the framework DRIP-MPU, which formulates this task as learning to retrieve a multiclass context for a frozen TabPFN. TabPFN serves both as a teacher guiding context retrieval and as the final in-context classifier. Starting from labeled positives and geometrically selected diverse pseudo-negatives, a Feedback-Guided Context Retriever learns from confidence-weighted out-of-fold TabPFN predictions and geometric supervision. The retrieved pseudo-labeled examples update the context for the next round of teacher feedback, iteratively refining the context without updating TabPFN's parameters. Experiments on 25 tabular datasets show that DRIP-MPU achieves the highest average AUC and positive-class F1 among the evaluated PU/MPU baselines across 5-, 10-, and 15-shot settings. In the 5-shot setting, it improves upon the strongest baselines on these metrics by 7.7% and 46.1%, respectively. Code: https://anonymous.4open.science/r/opal-forest-7c4d.