- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies how joint versus separate compression of multimodal observations affects prediction accuracy. Its central argument is that limiting the dimension of a representation alone does not guarantee meaningful compression if encoder and decoder complexity and precision are unrestricted. Additional constraints, such as Lipschitz regularity or finite communication rates, lead to more meaningful trade-offs.
Contributions:
1.The authors show that even a one-dimensional representation can approach the optimal prediction risk when model complexity and precision are unrestricted. For separate encoding, they characterize which modalities benefit from receiving nonzero-dimensional representations.
-
With Lipschitz-constrained reconstruction and for uniform data in dimensions, the paper derives the optimal scaling of scalar reconstruction error with the encoder--decoder Lipschitz budget, supported by matching lower and upper bounds.
-
A two-modality example shows that joint encoding can achieve a better asymptotic prediction rate than separate encoding, although the optimal joint rate remains unknown.
-
The paper analyzes prediction risk under Gaussian models and finite communication budgets, showing how performance depends on the structure of the prediction task.
-
Several theoretical examples and controlled experiments illustrate the predicted behavior under different resource constraints.
Critical strengths
-
The paper clearly distinguishes representation dimension from actual compression, emphasizing the need to constrain precision, complexity, or communication rate.
-
The matching bounds for scalar reconstruction under a Lipschitz constraint provide a precise characterization of how reconstruction quality scales with the available resources.
-
The example demonstrates that joint encoding can outperform separate encoding under the same overall resource constraint.
-
The paper connects nonlinear representations, Lipschitz constraints, and finite-bit compression within a common prediction framework.
Critical weaknesses
-
Very difficult to read. The paper needs better presentation, especially for those who are not familiar with this line of research.
-
The paper builds on existing scalar-coding results. It should more clearly explain which contributions go beyond prior work, particularly regarding general data distributions and modality selection.
-
The lower bound for scalar reconstruction is central to the paper. The authors should clarify the treatment of degenerate encoders, atoms in the encoded distribution, and the geometric assumptions behind the bound.
-
The paper establishes an advantage for joint encoding but does not prove that its achievable rate is optimal. A matching lower bound would strengthen the result.
-
A Lipschitz constraint does not directly limit network size or parameter magnitudes. The paper would benefit from analyzing how model complexity grows as prediction error decreases.
-
Broader comparisons and tests across different distributions and prediction tasks would provide stronger evidence of the results' practical relevance.
-
The authors should explain more clearly how prediction-task structure, covariance conditioning, and bit allocation affect the reported rates.
Questions for authors
See all the weaknesses above and additionally,
- How do the dimension-only results differ from existing scalar-coding methods?
- How does the required network size scale with the Lipschitz budget and target error?
- What assumptions are essential for the finite-bit Gaussian rate results?