- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes PhysWave for recovering rPPG signals from facial videos. AMWD combines multiple learnable wavelet filters according to the input and decomposition level, producing five streams comprising the original features and decomposed components. SOCA models dependencies within each stream, while TSCI uses temporal spectral similarity to guide cross-component feature exchange before waveform fusion.
The authors evaluate the method on five benchmarks through intra-dataset experiments and single-source and multi-source transfer, with VIPL-HR additionally used as a training source. The paper reports competitive heart-rate estimation results, particularly for transfer to MR-NIRP-Car, and includes component ablations and qualitative analysis.
Critical strengths
- The paper proposes an adaptive multi-expert wavelet decomposition that allows temporal filtering to vary across recordings. By conditioning the mixture of learnable filters on the input and decomposition level, AMWD provides greater flexibility than applying a fixed wavelet basis to every video. On MMPD, the adaptive design reduces MAE from 3.04 bpm for the best fixed-wavelet alternative to 2.81 bpm. This supports the usefulness of the complete adaptive decomposition module within the proposed architecture.
- The paper proposes a framework that combines intra-component refinement with spectrum-guided cross-component interaction. SOCA refines the spatiotemporal features within each component, and TSCI exchanges information according to their spectral relationships, providing a concrete mechanism for combining cues across scales. Across three separate source-to-MR-NIRP-Car transfer settings, the complete model reports average MAE and RMSE of 5.96 and 9.68 bpm, improving over the strongest competing averages in Table 3 by 31.7% and 24.8%, respectively. These results suggest practical value under changes in acquisition conditions.
Critical weaknesses
- The novelty needs more precise positioning relative to closely related methods. rPPG-TFCL already incorporates wavelet decomposition into hierarchical rPPG attention. More directly, SOCA’s orthogonally initialized projection and head-wise scaling are closely related to SOatten, while TSCI’s FFT-magnitude descriptors and scaled spectral queries and keys are closely related to FSatten. These mechanisms were introduced in Revisiting Attention for Multivariate Time Series Forecasting (AAAI 2025), which is not discussed in the paper. The component-level application, gating, and normalization may be useful adaptations, but the paper should explain their additional contribution more clearly. AMWD offers a clearer potential distinction, but its comparison with fixed wavelets changes filter learnability, expert count, and input-dependent routing together. The current ablations therefore leave uncertainty about which choices drive the improvement. TSCI’s MAE ablation gain is also small, and variation across runs would help establish its reliability. These limitations reduce confidence in the contribution of individual mechanisms, although the complete framework shows encouraging results.
- The training loss in the released code differs from the description in the paper. In PhysWave_Loss, the differentiable loss_distribution_kl returned by the frequency-loss function is not included in the total loss. The included HR_loss instead constructs distributions from detached predictions through NumPy operations, so it provides no gradient to the network. Consequently, the distribution-consistency term does not directly regularize training in this implementation. The authors should confirm whether this code produced the reported results and explain which objective was actually optimized. This matters for reproducing the experiments and interpreting the claimed training constraint.
Questions for authors
- What are the substantive technical differences between SOCA/TSCI and SOatten/FSatten? Which differences provide benefits beyond adapting these mechanisms to rPPG components?
- With the remaining architecture held fixed, how does AMWD compare with a learnable single filter pair, a learnable four-expert bank with input-independent mixing, and an input-routed bank with random initialization? Please also report variation across runs for the main ablations, particularly the small TSCI gain.
- Does the released PhysWave_Loss correspond to the reported experiments? How should the unused differentiable KL term and the detached HR_loss be understood? Please specify the objective used to produce the reported results and whether resolving this discrepancy changes the main findings.
3 comments
Response (Part 2)
TSCI. Beyond deriving queries and keys from frequency spectra, TSCI contains two rPPG-specific designs that make cross-component exchange effective:
- Spectral Normalization: Before the affinity is computed, temporal centering removes the DC term (skin tone and illumination baseline), log compression suppresses dominant motion peaks, per-channel spectral-mean normalization equalizes the energy scales of the components, and L2 normalization of the queries and keys turns the affinity into a comparison of spectral shapes rather than of energy.
- Diagonal self-retention bias: A learnable non-negative bias is added to the affinity, preserving each component's sub-band identity during exchange.
In addition, a single affinity per clip and head is shared over all spatiotemporal positions and applied through a zero-initialized gated residual, so the exchange starts from the identity mapping.
Table S9. Ablation of the rPPG-specific designs in TSCI on MMPD. Spec. norm: per-channel spectral-mean normalization; Q/K norm: L2 normalization of queries and keys; Diag. bias: diagonal self-retention bias.
| Variant | Spec. norm | Q/K norm | Diag. bias | MAE↓ | RMSE↓ | r↑ |
|---|---|---|---|---|---|---|
| w/o TSCI | – | – | – | 2.88 | 6.43 | 0.88 |
| No normalization | ❌ | ❌ | ✓ | 3.14 | 6.80 | 0.86 |
| Q/K norm only | ❌ | ✓ | ✓ | 3.61 | 8.17 | 0.80 |
| Spec. norm only | ✓ | ❌ | ✓ | 3.46 | 8.10 | 0.79 |
| No diag. bias | ✓ | ✓ | ❌ | 3.98 | 9.05 | 0.77 |
| Full TSCI | ✓ | ✓ | ✓ | 2.81 | 6.33 | 0.88 |
Every variant that lacks either normalization or the diagonal bias is worse than removing TSCI entirely (MAE 3.14–3.98 vs. 2.88), whereas Full TSCI improves to 2.81. These designs are therefore what make spectral exchange beneficial for rPPG components. The two normalizations are complementary: using only one is even worse than using neither, because the remaining nuisance factor (inter-component energy scale or descriptor norm) then dominates the affinity. Adding the diagonal bias reduces MAE/RMSE by 29.4%/30.1%.
In summary, what PhysWave contributes beyond spectrum- or orthogonality-based attention is (i) the pre-orthogonal routing and post-orthogonal gating that make the orthogonal projection effective in SOCA, and (ii) the spectral normalizations and the diagonal self-retention bias that make spectral exchange beneficial in TSCI. Built on the input-adaptive decomposition, they allow PhysWave to refine each scale and exchange information across scales without erasing the sub-band structure of the decomposition. Thank you again for this suggestion; we will highlight these points in the revised version.
Response (Part 3)
W1&Q2. AMWD compared with a learnable single filter pair, a learnable four-expert bank with input-independent mixing, and an input-routed bank with random initialization; relation to rPPG-TFCL; variation across runs.
R2. For the complete control table (Table S1) and the discussion of rPPG-TFCL, please see our response to Reviewer XLCD, W1&Q1 (a) → R1; for the per-seed results (Table S6) and the cross-dataset TSCI ablation (Table S7), please see W2&Q2 (b) → R4 and W2 (c) → R5 there.
We ran all three requested controls with the remaining architecture fixed. In summary:
- Learnable single filter pair: The best learnable single pair (db6) reaches MAE/RMSE of 2.97/6.41, but the four learnable single experts span MAE 2.97–3.77, so a single basis remains fragile.
- Learnable four-expert bank with input-independent mixing: It reaches 3.34/8.07 with uniform weights and 2.98/6.75 with globally learned weights, i.e., no better than the best single pair; expert diversity alone does not help.
- Input-routed bank with random initialization: It reaches 3.66/8.26, versus 2.81/6.33 for AMWD; wavelet initialization reduces MAE/RMSE by 23.2%/23.4%.
- Relation to rPPG-TFCL: rPPG-TFCL uses a single fixed wavelet basis for every video, whereas AMWD uses four learnable, wavelet-initialized experts with input-dependent routing; PhysWave attains lower RMSE than rPPG-TFCL in all four cross-dataset settings of Table 2.
- Variation across runs: Over three paired seeds on UBFC-rPPG, the full model outperforms w/o AMWD, w/o SOCA, and w/o TSCI in all 9 (variant, seed) pairs on both MAE and RMSE; without TSCI, MAE/RMSE rise from 0.45 ± 0.05/0.67 ± 0.04 to 0.53 ± 0.02/0.84 ± 0.05, and the TSCI gain is larger under domain shift (RMSE −10.7% to −28.6%).
Therefore, input-dependent routing over a wavelet-initialized expert bank is the key factor behind AMWD, and the component-level improvements are consistent across runs.
W2&Q3. Clarifying the use of the loss function in the released PhysWave code.
R3. Thank you for your attention to the implementation details and for your rigorous attitude. Our contribution lies in the PhysWave architecture (AMWD, SOCA, and TSCI). For the loss function, we directly use the publicly available implementation of RhythmFormer [2] without modification, so that the comparison with this closely related baseline reflects the architectural difference. The loss computed in the code is:
L = 0.2 · L_time + 1.0 · L_CE + 1.0 · L_HR
where L_time is the negative Pearson loss, is the frequency-domain cross-entropy loss, and is a heart-rate distribution term.
- Role of L_HR: As in RhythmFormer [2] , whose Sec. 3.4 describes heart-rate constraints used "to select checkpoints", is computed from detached predictions and serves as a heart-rate constraint for checkpoint selection on the validation split. It does not provide gradients. The network parameters are therefore optimized by:
L_opt = 0.2 · L_time + 1.0 · L_CE
- KL term: The
loss_distribution_klvalue returned by the frequency loss is not included in the training objective. - Impact on the main findings: All reported results were produced with this implementation, and all PhysWave variants and ablations use the same objective, so the main findings are unchanged.
- We have clarified the training objective and the role of in Sec. 4.1, and we will carefully organize and document the loss code in the released repository.
We thank you again for the constructive feedback. If our responses resolve your concerns, we would be grateful if you would consider further strengthening your support; we are happy to answer any additional questions.
[2] RhythmFormer: Extracting Patterned rPPG Signals Based on Periodic Sparse Attention, Pattern Recognition 2025
Response (Part 1)
Thank you for your positive assessment and valuable suggestions. Your professional insights have helped us gain a deeper understanding of our work and sharpen its positioning. For the question that overlaps with those of Reviewer XLCD (Q2), we point to the detailed responses posted under Reviewer XLCD's review and summarize the conclusions.
R1. Thank you for this professional suggestion. In essence, SOCA and SOatten [1] share a related idea of orthogonal projection, and TSCI and FSatten [1] share the idea of measuring relations through frequency spectra. However, our modules go well beyond these shared ideas. Our main contribution is a component-centric formulation for weak, interference-dominated rPPG signals: an input-adaptive multi-scale decomposition of facial-video dynamics, followed by refinement within each decomposed component (SOCA) and spectrum-guided exchange across components (TSCI). Within this formulation, SOCA and TSCI contain the following rPPG-specific designs, which we will highlight and strengthen in the revision, together with a discussion of [1].
SOCA. SOCA builds the orthogonal projection into a three-step design that lets the projection play a larger role:
Table S8. Ablation of the three SOCA steps (MAE / RMSE / r). Pre-orth.: top-k routing on the TDC queries and keys; Orth. init. : orthogonally initialized projection; Post-orth.: learnable gate.
†The projection is kept but randomly initialized.
Each step makes the orthogonal projection more effective: the post-orthogonal gate reduces MAE on MMPD from 3.14 to 2.81 and raises r on real-world driving videos (MR-NIRP-Car) from 0.23 to 0.39; orthogonal initialization reduces MAE from 3.00 to 2.81 ( Table 8); and adding the pre-orthogonal routing to the projection changes MAE from 3.31 to 3.14. Overall, the full design reduces MAE from 4.01 without SOCA to 2.81.
[1] Revisiting Attention for Multivariate Time Series Forecasting, AAAI 2025.