- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes PACT which is a framework for learning joint distributions from partially paired data by combining relation learning with empirical marginal constraints. The authors analyzed how marginal estimation affects the learning objective and showed that additional unpaired observations can change the direction of the leading bias. Also, they proposed a correction to reduce first-order objective bias. The method is evaluated on controlled experiments and applications involving conditional prediction and risk assessment.
Critical strengths
I think the paper addresses an interesting problem, especially, when paired observations are limited but additional marginal information is available. The proposed framework is well motivated, and the overall structure of the paper is clear and easy to follow.
One of the main strengths is the theoretical analysis, particularly how the authors study the effect of empirical marginal constraints on the learning objective and parameter estimation. I also appreciate that the paper provides a correction method based on this analysis and investigates its effectiveness.
Another positive point is the experimental evaluation. The authors considered several settings, including synthetic experiments, Swiss roll, Colored MNIST, finance, and queueing. The experiments showed useful evidence supporting the theoretical findings. I also appreciate that the authors report cases where the proposed correction does not improve performance, which helps clarify the limitations of the method.
Critical weaknesses
My main concern is the reproducibility of the experiments. Although the paper provides detailed experimental settings and pseudocode in the appendix, I could not find the implementation code in the submitted materials. Since the method involves several numerical procedures and different experimental settings, I think providing the code would be important for independently verifying the results.
Another concern is the practical effectiveness of the proposed bias correction. While the theoretical analysis shows that the correction removes the first-order bias in the expected objective, the experimental results do not consistently demonstrate improvements in estimation performance. For example, Table 2 shows that the correction can perform worse than the uncorrected method under some settings. I think the authors should explain more clearly when the correction is beneficial in practice.
I also think the authors could further clarify the novelty of the framework compared to existing methods based on entropic optimal transport and calibrated likelihood. In addition, the theoretical guarantees are mainly established under finite discrete support assumptions, while some of the applications involve more complicated settings. It would be useful to discuss how these theoretical findings extend to the practical experiments.
Also, I had some problems with reading the math in the paper. I am not an expert in that field and not aware about the standard notations but I think that it is better to make the theory and proofs more readible and use better notations. Regarding the derivations and etc, I did not find major problems, but I did not spend too much time on it and might have some misunderstandings.
Questions for authors
-
Could the authors provide the implementation code and scripts needed to reproduce the main experiments?
-
The proposed correction theoretically removes the first-order expected objective bias, but it does not consistently improve the experimental results. Could the authors explain under which conditions the correction is expected to provide a practical benefit?
-
In Table 3, the mixed-sample likelihood approach does not outperform the paired-only alternatives. Could the authors explain the advantage of incorporating unpaired observations directly during relation learning instead of using them only for final marginal calibration?
-
Could the authors clarify the main methodological novelty of PACT compared with existing entropic transport and calibrated-likelihood approaches?
-
To what extent can the theoretical bias analysis be extended to continuous distributions and neural network models?
5 comments
Thank you to the authors for the detailed rebuttal and additional clarifications. I appreciate their efforts to address my concerns, particularly by releasing the implementation code and providing further experimental evidence.
After reviewing the rebuttal, manuscript, and released code, I still have a few remaining concerns:
-
Reproducibility: The code release substantially addresses my original concern. However, I could not locate the scripts, configurations or results corresponding to the new Swiss-roll experiment with 4,096 training pairs reported in the rebuttal. Since this experiment provides important additional evidence for the proposed correction, it would be useful to make it reproducible.
-
Practical effectiveness: Although the new results are encouraging, the original experiments show that the bias correction does not consistently improve estimation performance. I remain somewhat unconvinced about when and how reliably the correction provides practical benefits.
-
Theoretical scope and novelty: The clarification of the contribution relative to existing entropic OT methods is helpful. However, the theoretical analysis remains largely restricted to fixed finite supports, and its implications for the more complex applications are not fully established.
Overall, I recognize the interesting theoretical contribution and appreciate the authors' efforts during the rebuttal. Nevertheless, I believe the remaining concerns, particularly regarding the reproducibility and practical effectiveness of the correction, prevent me from increasing my score.
I therefore maintain my rating of 2 (weak rejection).
Thank you for pointing out the missing materials for the 4,096-pair Swiss-roll experiment. We have added a standalone supplement to the original anonymous repository. It includes data generation, training and evaluation scripts, frozen settings, and per-query scores for all ten confirmation archives. After unpacking the ZIP, run python3 reference/verify_reference_results.py from the supplement directory to reproduce the development selection and paired comparisons from the saved scores without a GPU.
https://anonymous.4open.science/r/pact-review-7d2k/conditional/swiss_roll_large_archive/README.md
With equal training-time budgets and batch size 64, the correction lowered matching log loss by 2.4% relative to uncorrected training, with improvements in all ten archives. The prespecified comparison with full-archive training remained inconclusive (mean loss difference +0.011, paired 95% CI [−0.003, +0.025]).
Our analysis shows that changing the ratio of paired to unpaired observations can reverse the leading bias in learned dependence even when the true relationship stays fixed. This makes the sampling ratio a training choice worth validating on held-out pairs. The theoretical guarantee applies to fixed finite supports. The Swiss-roll experiment tests the correction’s predictive effect in a synthetic neural matching task.
Thank you for your feedback. PACT learns correspondence from limited pairs and uses marginal calibration to construct a joint distribution. With 128 pairs on Swiss roll, PACT achieved a conditional MMD of 0.108, versus 0.455 for native EBiEOT-GMM and 0.415 for a paired-only MDN, leading in all three confirmation replicates (Figure 4, Table 14). In simulated hedging, all methods shared the same empirical marginals. PACT achieved a dependence error of 0.1016, versus 0.1095 for inverse EOT and 0.1192 for projected pair diffusion. Both prespecified comparisons favored PACT after adjustment for multiple comparisons (Tables 5 and 32).
What is new. EBiEOT already combines paired and unpaired data, and our paired-only likelihood aligns with Sei and Yano’s profile likelihood under identical scores and exact calibration. Our contribution is to predict how the ratio of these two data sources changes what the model learns. When the same pairs provide supervision and help estimate the minibatch marginals, adding independent endpoints changes the weights on the variance and covariance terms in the leading bias. For example, in a binary model with uniform marginals and true correlation 0.7, the expected objective favors stronger dependence to first order with pairs alone. Adding one independent endpoint per pair on each side reverses that shift. Figure 3 tests the predicted bias directions. This analysis also explains why the split-jackknife preserves the sampling ratio in each half.
https://arxiv.org/abs/2410.02628
https://link.springer.com/article/10.1007/s10463-026-00995-8
When the correction helps. In a new synthetic Swiss-roll candidate-matching study with 4,096 training pairs per archive, the original correction reduced held-out log loss by 2.4% relative to uncorrected training at batch size 64, improving all ten independent confirmation archives under equal training-time budgets and development-based checkpoint selection. At matched 300-second checkpoints, the gain persisted in nine of ten archives, with lower mean top-1 accuracy. The primary comparison with full-archive training remained inconclusive (loss difference +0.011, paired 95% CI [−0.003, +0.025]).
Unpaired data in training. With final calibration held fixed in Table 3, mixed likelihood improved mean NLL over paired likelihood on Colored-MNIST, where paired contrastive learning performed best. Paired likelihood had the lowest mean error on Swiss roll and in finance. The framework lets practitioners choose the training objective for the task and still use unpaired data to calibrate the final coupling.
Theory and reproducibility. Theorem 1 applies to smooth neural scores on fixed finite supports under its regularity conditions. Continuously sampled supports require bounds that remain valid as the support grows. We will clarify the distinct roles of marginal and parameter curvature in the proof. The anonymous repository provides code, configurations, and run scripts for the finite-support studies and all four applications. The top-level README links to the setup and run instructions.
https://anonymous.4open.science/r/pact-review-7d2k/README.md