acceptodds
Under review as a conference paper at ICLR 2027

JEPA-MUSIC: Self-Supervised Predictive Subspace Learning for Multiple Signal Classification

Abstract

Direction-of-arrival (DoA) estimation is fundamental to modern wireless communication and sensing systems. While the classical multiple signal classification (MUSIC) algorithm offers high angular resolution and clear interpretability, its performance degrades under limited snapshots, low signal-to-noise ratios (SNRs), and coherent sources due to empirical covariance breakdown. Conversely, data-driven methods mitigate such degradation but typically demand large DoA-labeled datasets and yield less transparent inference. To bridge this divide, we propose JEPA-MUSIC, a model-based self-supervised learning framework that integrates representation learning with subspace-based MUSIC inference. Specifically, JEPA-MUSIC constructs paired observations from the same signal scene and employs a tailored joint-embedding predictive architecture (JEPA) to learn to recover a structured, MUSIC-compatible covariance without requiring ground-truth DoA labels. During inference, this recovered covariance directly feeds into standard MUSIC for spatial-spectrum search, preserving its rigorous model-based interpretability. Extensive evaluations across diverse sensing conditions demonstrate that JEPA-MUSIC substantially outperforms classical MUSIC in challenging regimes and matches or surpasses strong data-driven baselines, while achieving higher subspace fidelity, lower tail failure rates, and superior sample efficiency.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
DoA estimation, Multiple Signal Classification, Self-Supervised Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2CYFOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper designs a JEPA-based ML model for DoA estimation that interfaces directly with legacy DoA solvers. The model produces an increased quality estimate of the measurement covariance matrix (CM). The JEPA is trained with a combination of low-quality masked CMs and paired high-quality CMs and includes some regularization measures that enforce the output to be Hermitian and positive semidefinite (e.g., CM-like). This is in contrast to most ML-based DoA approaches that aim to learn the data-to-DoA map directly in a supervised learning setting. The premise of the paper is that during training a high-quality CM can be obtained by improving the SNR or by computing the CM over a larger number of data windows. Experiments show that the proposed JEPA-MUSIC outperforms other previously proposed ML models for DoA as well as the legacy MUSIC method.

Critical strengths

The use of JEPA as a denoiser of CMs appears novel.

The approach outperforms other DoA based approaches, seemingly due to the use of high-quality CMs during training. Since both low-quality and high-quality CMs are used in training but only low-quality CMs are used during testing, the JEPA could be interpreted as a denoiser that operates in the latent space.

Alternatively, one could argue that the JEPA is learning the manifold containing the CMs corresponding to the different target configurations

Critical weaknesses

The proposed model appears to be the only approach tested that relies on high-quality CM estimates to improve performance. It is not clear from the description whether other ML-based models are also benefiting from high-quality (target observations) CMs during training.

If one desires to use a JEPA, it could be designed to learn the signal subspace directly instead of the CM. These are two different manifolds that would share a parametrization (the source angles), and the signal subspace will be easier to learn than the CM. In fact, given that the first step in MUSIC is to estimate the signal subspace, such a change would subsume that first step and move directly into the pseudospectrum computation. In the same vein, standard ML-based DoA approaches aim to learn the parametrization of such manifold directly, so it is not clear why it is beneficial to try to learn the manifold of CMs instead of its parametrization.

Alternatively, one could interpret the proposed JEPA model as impleementing a denoiser of the low-SNR CM, and many existing approaches to low rank matrix denoising would be responsive to this problem as well.

There is not much argument for the setup in which such training is possible but the deployment of standard DoA cannot benefit from the same measures (e.g., where we would engage in acquiring large amounts of measurements for the training process in order to reduce the number of windows or entries of the matrix that need to be computed when performing DoA estimation).

Questions for authors

Can you confirm whether baseline models use the target observations used by JEPA-MUSIC in any way during training and testing? Can you comment on whether there would be any drawbacks in aiming to learn the signal subspace directly instead of the CM? Were approaches to denoising considered when attempting to obtain a high-quality CM estimate from a low-quality input? (e.g., diffusion or autoencoder models) Do you see shortcomings to the interpretation of your framework as a CM denoiser?

It seems that the real/imaginary mixed expression in line 17 could be written simply in terms of the difference of the complex matrices? The Frobenius norm already uses the magnitude of the matrix entries in its computation.

2 comments
Reviewer_2CYFOpenReview ↗

Thank you for the detailed responses.

Thank you for the detailed responses.

The first additional experiment for W1 and Q1 and the experiments for W3, Q3 and Q4 indeed show that denoising alone is not sufficient to match the performance of the proposed approach, and that may be because the denoiser is agnostic to the role the CM will play in DoA. However, the second experiment for W1/Q1 (if I understand correctly) uses the same number of training scenes but at a different noise level than JEPA. Or do the results in Table R2 have JEPA-MUSIC use exactly the same data (snapshots and noise levels) that the baselines use?

For W2 and Q2, my comment refers to whether it would be better for the JEPA to learn E directly without having to build R on the way. As the authors point out, orthogonalization can be enforced post-learning just like CM structure is currently enforced; and it is not clear to me how the eigenvalues are used in the current method. As detailed in the discussion with reviewer jfpc, it is not clear that the covariance loss is helpful to the method, and the authors are now recognizing that the projector loss seems to be the main learning signal.

While the description in Appendix D.1 now provides full detail in the training/testing data generation, I wonder if it is necessary to have exactly matching pairs of target and context observations. Have the authors tested the effect of having independently generated target configurations for these two portions of the dataset? This is related to my concern W4, if we are expecting to see matching configurations during training and deployment. It is again not clear to me where one could take benefit of such well-conditioned data acquisition for training purposes only.

I would find the numerical results for the proposed approach more compelling if the baselines also benefitted from the existence of high-quality target observations in order to clarify that the gains obtained come from the method and not the additional data. The authors have a valuable point in that even if the performances of all the methods were comparable in that setting their proposed approach requires no label/DoA ground truth information. However, I think one fair way to provide a comparison is to allow for the exact same data to be used in all methods, particularly since it is now clear that the context and target observations are closely paired and it seems obvious that other methods would benefit from having those high quality observations be included during inference for each data sample. Giving the high-quality target observations to the other methods during inference would be the most lenient alternative to baselines; alternatively, you could consider using the same quality of observations in the target and context datasets to see if it is the data quality and not the quantity that gives the proposed methods its benefits.

I do recognize that some of the tests I am proposing here go beyond the scope of the paper submitted, but I do think the authors can provide a more even keeled comparison of the method proposed to baselines, particularly since the proposed method requires not only a more sophisticated model but also additional high-quality data/observations.

Reviewer_2CYFOpenReview ↗

Supervision from DoA labels vs high-quality observations

Thank you for the additional response. This synchronization does give me an idea for how baseline methods could leverage the same target observations as the proposed method following a semi-supervised learning (SSL) approach: (a) run MUSIC on the target observations to obtain DoA labels, (b) use the target observations and SSL labels to train the baseline. Depending on the quality of the observations, the SSL labels may or may not be accurate. There is a question with respect to the computational cost (e.g., how much compute it will take to create the SSL labels vs. the to train the JEPA). I share this idea while acknowledging that it's too late in the discussion period to try it out, so I won't consider its implementation or lack thereof as a factor in my review.

I also recognize that the other reviewers do not seem to share my concerns on the feasibility of a two-stage data acquisition setting (high-quality targets and low-quality context), which was the main reason for my rating. Thus I will increase my rating accordingly.

Reviewer_VqE3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes JEPA-MUSIC for direction-of-arrival estimation from limited, noisy array observations. It pairs a low-quality context observation with a higher-quality observation of the same scene. Training combines latent prediction, covariance alignment, and signal-subspace alignment. A structured head produces a covariance matrix that is passed to classical MUSIC at inference. The analysis relates covariance errors to subspace and MUSIC-spectrum-denominator errors under an eigengap condition. Experiments cover synthetic signals and DeepMIMO-derived observations, including direct transfer without fine-tuning.

Critical strengths

The method addresses a useful problem: improving the covariance supplied to MUSIC without requiring ground-truth DoA labels for training. The structured output keeps the final estimation step easy to interpret. The empirical results are also encouraging. At the default setting, JEPA-MUSIC improves both RMSE and failure rate over the reported baselines for noncoherent and coherent sources. The DeepMIMO direct-transfer results provide further evidence that the learned representation is useful beyond the original synthetic test set.

Critical weaknesses
  1. Equation (19) combines the JEPA loss with direct covariance and projector losses. The latter two can train a covariance-recovery model using the same observation pairs without a target-embedding prediction loss. That alternative would also avoid DoA labels. It is therefore important to establish what the JEPA objective adds. Table 5 removes the predictor or EMA, but neither ablation removes the latent loss itself. In the coherent setting, the mean RMSE changes from 2.915 to 3.041 without the predictor and to 3.093 without EMA. These are modest changes relative to the reported variation across seeds, and some other metrics improve slightly. This does not show that JEPA is ineffective, but it leaves its contribution uncertain. A direct test would keep the context encoder, predictor, covariance head, preprocessing, and training pairs fixed, and set the latent-loss weight to zero. The target encoder would then be unnecessary. This would separate the value of latent prediction from the value of learning covariance and subspace targets.

  2. Appendix D.1 uses target observations with 512 snapshots at 20 dB SNR. Context observations have 32 snapshots, with training SNR sampled from [-5, 5] dB and a default test SNR of 0 dB. The target is needed only during training, but the practical value of the method still depends on obtaining such pairs. Appendix B.1 gives reasonable ways to construct them, including longer stationary observation windows, cleaner repeated measurements, and adding noise to the context. However, the experiments do not show how performance changes when those conditions are less favorable. In particular, adding noise can degrade a context observation; it does not make the target measurement cleaner. This also limits the interpretation of Figure 4. It demonstrates efficiency in the number of training scenes or pairs, but does not establish efficiency in the total measurement budget: each JEPA training pair includes an additional high-quality observation, while the supervised baselines use DoA labels. Both resources have costs. A target-quality sensitivity experiment would help establish how much of the reported advantage survives with more modest target observations.

Questions for authors
  1. How much of the reported improvement is specifically attributable to JEPA-based representation learning? Can the authors compare against a direct covariance and subspace regression model trained on the same context–target pairs, using comparable network capacity and optimization settings?

  2. How does the performance change when the target observations have the same SNR as the context observations, or when the target snapshot count is reduced? Can the authors evaluate sample efficiency under a matched total observation budget?

Reviewer_jfpcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes a JEPA-based MUSIC approach for the classic DoA estimation problem. The proposed method doesn't require the ground truth directions, instead, it is trained based on a set of higher-quality target snapshots paired with the context or observed snapshots. During training, JEPA is employed to predict the higher-quality embeddings from the context snapshots. The context snapshots first go through random masking and is encoded by the context encoder to get its latent representation. This latent representation is then taken by the predictor to predict the latent representation of the higher-quality snapshot pairs. A covariance head is added to process the latent representation and output a covariance proxy. The covariance proxy serves the loss functions of covariance fitting and projection fitting, along with the latent fitting loss. Experiments compare the proposed JEPA approach to MUSIC and deep learning based subspace methods. JEPA shows better RMSE when the SNR is low in few-snapshot scenarios.

Critical strengths

The paper is well-written and easy to follow. The background materials are sufficient and the problem is well-motivated. The JEPA idea is interesting and seems new in the field of DoA estimation. In addition, the usage of higher-quality snapshots and not using the ground-truth DoA seem to be useful in practice.

Critical weaknesses

In general I think this is an interesting work but there are several issues in the current form of the paper.

First, the evaluation is not complete and the most important cases are missing. If we look at Table 1 and Figure 3, these are scenarios with either low SNRs or few snapshots. For example, in Figure 3(c), although the SNR varies from -10 to 20, the snapshot is kept at 32. In Figure 3(d), the snapshot varies from 8 to 256 but the SNR is kept at 0 dB. These are known to be easy cases for deep learning based methods and we can see that MUSIC performs poorly. A good deep learning based approach should not only outperform MUSIC in low SNR (or easy region), but also matches the performance of MUSIC in the scenarios with both high SNR and/or snapshots. This will also allow us to understand whether the proposed method is optimal asymptotically like MUSIC. Another missing indicator is the CRLB. We should add the CRLB and compares it with JEPA.

Second, with 8 sensors, the model should be able to detect up to 7 sources but the evaluation only covers the case of 3 sources. A more comprehensive evaluation with different source numbers should be included to see whether JEPA is robust with different source numbers.

Third, the setup is not clear to the reader. For example, the information about the spacing between sources is missing. Is the result based on a fixed spacing or random spacing? If random spacing is applied, what is the distribution? Given that the RMSE is really sensitive to the setup, the setup should be made clear.

Fourth, λP=20\lambda_P=20 is much larger than λz=0.1\lambda_z=0.1. Does this mean that we actually don't need JEPA and we can already achieve good results with the projector and covariance fitting losses? The ablation on this is missing and it is not known whether JEPA is the main component that leads to better performance in low SNRs.

Finally, there are some missing references including a few seminal works on subspace learning (see the references below). Given these methods achieve SOTA results and given the proposed approach uses a projector (subspace) loss, a few papers in this direction should be included and discussed. Although their target applications are slightly different but their loss functions are the same as the projector loss and are directly applicable in classic DoA estimation.

[1] Chen, Kuan-Lin, and Bhaskar D. Rao. "Subspace representation learning for sparse linear arrays to localize more sources than sensors: A deep learning methodology." IEEE Transactions on Signal Processing 73 (2025): 1293-1308.

[2] Yu, Wentao, Khaled B. Letaief, and Lizhong Zheng. "Sensing for free: Learn to localize more sources than antennas without pilots." IEEE Journal on Selected Areas in Communications (2026).

Questions for authors
  1. Are we assuming that the number of sources is known or given? It is not clear whether JEPA also learns select the model order.
  2. Is there an ablation study on the loss components? For example, if we drop the projector loss, would it impact the RMSE? What if the latent representation loss is dropped? Would it still work?
  3. The setup in Table 3 seems to suggest that the model was trained with D=3 only. Would it be a fair comparison if the model only works for D=3 while MUSIC can work for all source numbers?
  4. What is the spacing distribution during training and inference? How does the model perform when the sources are closely spaced?
  5. How does the model perform in the cases with high SNRs and/or snapshots?
  6. What if the test condition has higher SNRs and snapshots than the target pairs (20 dB, Nt=512N_t=512)?

Comments on the paper

3 comments

Thanks for the detailed reply and for running the experiments I asked about. I have also read the revised paper.

The new ablation answers my question about the latent loss, thank you. Table 7 shows the difference between the full and the no-latent model is within the seed variation (2.915±0.694 vs 3.226±0.413 for coherent sources, 1.012±0.241 vs 0.882±0.105 for noncoherent), and Appendix E.1 says so as well. The loss that really matters is the projector loss. So I still have a concern here. The appendix now says the projector loss is the main learning signal, but the abstract, the contributions and the conclusion are unchanged and still present JEPA as the core of the method.

The target-quality results are helpful. They show the method still beats classical MUSIC with modest targets, which is good. But the main text now says the efficiency "does not hinge on high-quality targets", and I think that is too strong. In Table 3 the RMSE goes from 1.012 to 1.693 to 3.360 as the target gets worse. With same-SNR targets the coherent RMSE (4.427) is also higher than DeepCNN and DA-MUSIC in Table 1, but Table 3 only compares with classical MUSIC.

I have one more concern about Table 16. The supervised baselines have failure rates above 90% at 400 scenes, worse than raw MUSIC. Appendix E.9 already says SubspaceNet is limited by the training setting. Were the baselines tuned for the small-data case?

I appreciate the effort, and the new experiments make the paper more complete. But they also confirm my two concerns. The contribution of JEPA itself is small, and the gain over supervised methods relies on high-quality targets. So I'm keeping my score.

Area_Chair_8rj8OpenReview ↗

Please join the author-reviewer discussion

Dear all reviewers,

Our author and reviewer discussion will end soon in 18 November (AOE). If you have not yet joined the discussion, now it is the time to start it. We highly appreciate your time on discussing and exchanging thoughts to keep our ICLR standards. Many thanks!

Bests,
Your AC

Thank you for the response. I will maintain the original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.