- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes RePs, which utilizes the known response patterns to learn and measure the relevant signal representations. The core approach is to simultaneously transform the signals and the existing measurement values, to enhance the supervisory target for view generation. The model only receives the enhanced signals, without receiving the original measurement values or transformation parameters. This supervision can be used alone or combined with contrastive learning. It requires source samples with existing measurements, but does not need to add annotations for the enhanced view. This idea is applied to electrocardiogram time scaling, music pitch translation, and spatial audio rotation. The main contributions include a systematic comparison of the supervisory targets and the usage of the model. In the main experiments of freezing the encoder and re-training the nonlinear reader, directly predicting the transformed measurements is superior to predicting the original measurements and the transformation parameters separately. However, in the music pitch task, reusing the prediction head during training causes the ranking to reverse. This indicates that a supervision target suitable for learning the representational features that can be re-read is not necessarily the most suitable for directly reusing the trained predictor. The validation covers the response prediction of three types of signals and five clinical transfer tasks. RePs outperforms the baseline that does not use measurement supervision in all three response predictions. The spatial angle error is reduced from approximately 69.0° to 20.2°. In the clinical experiment using only 10% downstream training labels, its model achieves higher scores in four out of the five tasks than the average of four public encoders. However, simple supervision using the existing measurements also achieves similar results. Therefore, the experiments fully support the value of measurement supervision, but the additional benefits brought by the response patterns themselves still depend on the specific task.
Critical strengths
- It provides valuable empirical insights into the relationship between the supervision target and the model usage method. In the control group using the same source measurement, direct response supervision outperformed separate prediction of the original measurement and transformation parameters in the main nonlinear readout experiments in three domains, and six comparisons passed the Holm correction. However, on NSynth, when re-fitting the reader, RePs outperformed Factorized; when retaining the training prediction head, the ranking reversed. This indicates that "suitable learning of the representational features that can be re-read" and "suitable training of the predictor that can be directly reused" are not the same goal.
- The matching control shows the actual transfer value of measurement-assisted supervision. The clinical experiment matched pre-trained data, backbone networks, initialization seeds, and training budgets for each internal method, reducing the interference of resource differences on the results. When using only 10% of downstream training labels, RePs-SSL outperformed the matched SimCLR in five clinical tasks. PTB-XL AUROC increased from 0.662 to 0.836, and Chapman F1 increased from 0.621 to 0.873. This indicates that the benefits are not limited to enhancing the response prediction of the signal, but can also be transferred to downstream clinical tasks.
Critical weaknesses
The core design's independent contribution compared to ordinary measurement supervision has not been fully established. The paper has acknowledged in related work that the synchronous transformation of signals and labels belong to existing supervised enhancement. Therefore, the key is not only whether performance can be improved after adding measurement supervision, but also what additional value the design of responding patterns provides.
Table 19's matched clinical experiments show that merely predicting the original measurement's Identity has achieved a close mean. Its PTB-XL AUROC is 0.8329, and RePs-SSL is 0.8359; its Chapman F1 is 0.8774, which is higher than RePs-SSL's 0.8730. This table reports the paired confidence intervals relative to SimCLR, but does not fully quantify the incremental effect relative to these measurement supervision controls and its uncertainty. In Table 8, RePs does have advantages in ECG and spatial audio response prediction, but NSynth's Static error is 7.491 semitones, which is close to RePs' 7.502 semitones. The benefits have obvious task dependence.
These results do not deny the effectiveness of the method, but the most significant application benefits more strongly prove that "measurement supervision is useful", and are not sufficient to establish the independent value of the "response pattern objective". With the basic operation having a history, the paper has not clearly defined the applicable conditions for its additional benefits, nor has it fully extracted the empirical conclusions that are sufficient to carry the main innovation. This is the decisive consideration for my decision to reject the paper, rather than considering the method to be wrong or lacking ablation studies.
Questions for authors
In the clinical migration process, how much greater is the independent benefit of the response rule target compared to the ordinary measurement supervision?
Based on the results of the existing five random seeds, please provide the paired differences and 95% confidence intervals of RePs-SSL relative to Identity and Static. Can these results support that correctly responding to the target has additional value, rather than merely proving that adding measurement supervision is effective?