- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper designs a JEPA-based ML model for DoA estimation that interfaces directly with legacy DoA solvers. The model produces an increased quality estimate of the measurement covariance matrix (CM). The JEPA is trained with a combination of low-quality masked CMs and paired high-quality CMs and includes some regularization measures that enforce the output to be Hermitian and positive semidefinite (e.g., CM-like). This is in contrast to most ML-based DoA approaches that aim to learn the data-to-DoA map directly in a supervised learning setting. The premise of the paper is that during training a high-quality CM can be obtained by improving the SNR or by computing the CM over a larger number of data windows. Experiments show that the proposed JEPA-MUSIC outperforms other previously proposed ML models for DoA as well as the legacy MUSIC method.
Critical strengths
The use of JEPA as a denoiser of CMs appears novel.
The approach outperforms other DoA based approaches, seemingly due to the use of high-quality CMs during training. Since both low-quality and high-quality CMs are used in training but only low-quality CMs are used during testing, the JEPA could be interpreted as a denoiser that operates in the latent space.
Alternatively, one could argue that the JEPA is learning the manifold containing the CMs corresponding to the different target configurations
Critical weaknesses
The proposed model appears to be the only approach tested that relies on high-quality CM estimates to improve performance. It is not clear from the description whether other ML-based models are also benefiting from high-quality (target observations) CMs during training.
If one desires to use a JEPA, it could be designed to learn the signal subspace directly instead of the CM. These are two different manifolds that would share a parametrization (the source angles), and the signal subspace will be easier to learn than the CM. In fact, given that the first step in MUSIC is to estimate the signal subspace, such a change would subsume that first step and move directly into the pseudospectrum computation. In the same vein, standard ML-based DoA approaches aim to learn the parametrization of such manifold directly, so it is not clear why it is beneficial to try to learn the manifold of CMs instead of its parametrization.
Alternatively, one could interpret the proposed JEPA model as impleementing a denoiser of the low-SNR CM, and many existing approaches to low rank matrix denoising would be responsive to this problem as well.
There is not much argument for the setup in which such training is possible but the deployment of standard DoA cannot benefit from the same measures (e.g., where we would engage in acquiring large amounts of measurements for the training process in order to reduce the number of windows or entries of the matrix that need to be computed when performing DoA estimation).
Questions for authors
Can you confirm whether baseline models use the target observations used by JEPA-MUSIC in any way during training and testing? Can you comment on whether there would be any drawbacks in aiming to learn the signal subspace directly instead of the CM? Were approaches to denoising considered when attempting to obtain a high-quality CM estimate from a low-quality input? (e.g., diffusion or autoencoder models) Do you see shortcomings to the interpretation of your framework as a CM denoiser?
It seems that the real/imaginary mixed expression in line 17 could be written simply in terms of the difference of the complex matrices? The Frobenius norm already uses the magnitude of the matrix entries in its computation.
2 comments
Supervision from DoA labels vs high-quality observations
Thank you for the additional response. This synchronization does give me an idea for how baseline methods could leverage the same target observations as the proposed method following a semi-supervised learning (SSL) approach: (a) run MUSIC on the target observations to obtain DoA labels, (b) use the target observations and SSL labels to train the baseline. Depending on the quality of the observations, the SSL labels may or may not be accurate. There is a question with respect to the computational cost (e.g., how much compute it will take to create the SSL labels vs. the to train the JEPA). I share this idea while acknowledging that it's too late in the discussion period to try it out, so I won't consider its implementation or lack thereof as a factor in my review.
I also recognize that the other reviewers do not seem to share my concerns on the feasibility of a two-stage data acquisition setting (high-quality targets and low-quality context), which was the main reason for my rating. Thus I will increase my rating accordingly.
Thank you for the detailed responses.
Thank you for the detailed responses.
The first additional experiment for W1 and Q1 and the experiments for W3, Q3 and Q4 indeed show that denoising alone is not sufficient to match the performance of the proposed approach, and that may be because the denoiser is agnostic to the role the CM will play in DoA. However, the second experiment for W1/Q1 (if I understand correctly) uses the same number of training scenes but at a different noise level than JEPA. Or do the results in Table R2 have JEPA-MUSIC use exactly the same data (snapshots and noise levels) that the baselines use?
For W2 and Q2, my comment refers to whether it would be better for the JEPA to learn E directly without having to build R on the way. As the authors point out, orthogonalization can be enforced post-learning just like CM structure is currently enforced; and it is not clear to me how the eigenvalues are used in the current method. As detailed in the discussion with reviewer jfpc, it is not clear that the covariance loss is helpful to the method, and the authors are now recognizing that the projector loss seems to be the main learning signal.
While the description in Appendix D.1 now provides full detail in the training/testing data generation, I wonder if it is necessary to have exactly matching pairs of target and context observations. Have the authors tested the effect of having independently generated target configurations for these two portions of the dataset? This is related to my concern W4, if we are expecting to see matching configurations during training and deployment. It is again not clear to me where one could take benefit of such well-conditioned data acquisition for training purposes only.
I would find the numerical results for the proposed approach more compelling if the baselines also benefitted from the existence of high-quality target observations in order to clarify that the gains obtained come from the method and not the additional data. The authors have a valuable point in that even if the performances of all the methods were comparable in that setting their proposed approach requires no label/DoA ground truth information. However, I think one fair way to provide a comparison is to allow for the exact same data to be used in all methods, particularly since it is now clear that the context and target observations are closely paired and it seems obvious that other methods would benefit from having those high quality observations be included during inference for each data sample. Giving the high-quality target observations to the other methods during inference would be the most lenient alternative to baselines; alternatively, you could consider using the same quality of observations in the target and context datasets to see if it is the data quality and not the quantity that gives the proposed methods its benefits.
I do recognize that some of the tests I am proposing here go beyond the scope of the paper submitted, but I do think the authors can provide a more even keeled comparison of the method proposed to baselines, particularly since the proposed method requires not only a more sophisticated model but also additional high-quality data/observations.