acceptodds
Under review as a conference paper at ICLR 2027

RAE-PPG: Duration-Grounded Retain-and-Extend Pretraining for PPG Foundation Models

Abstract

Signal features derived from photoplethysmography (PPG) require different signal durations to characterize. Existing PPG foundation models treat duration as a pretraining or evaluation condition rather than using the different durations required by PPG features to organize self-supervision. We hypothesize that self-supervision should expand with signal duration, allowing one encoder to progressively acquire additional features while preserving and reusing earlier learning. We introduce Retain-and-Extend PPG (RAE-PPG), which trains a single Transformer encoder through successive 10 s, 30 s, and 240 s input-duration stages, introducing supervision for additional signal features as longer observations become available. The encoder is partitioned into duration-specific parameter groups, allowing later stages to access groups learned earlier while updating only the group assigned to the current stage. Retained-target supervision continues supervision for selected earlier features at longer durations, encouraging them to remain decodable. Direct decoding from the final encoder shows that earlier features remain recoverable from longer-input representations, while later-stage features show higher mean decoding performance at their introduction durations. Controlled comparisons further show that prior-stage learning provides a better basis for learning newly introduced features at both transitions. Across 18 tasks from eight datasets, the final frozen encoder achieves the best observed score on 12 tasks compared with five existing PPG foundation models.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Photoplethysmography (PPG), Foundation Models, Self-Supervised Learning, Time-Series Representation Learning, Multi-Duration Learning

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_qYXPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a retain-and-extend supervision framework to learn informative PPG representations from different signal durations. In RAE-PPG, training proceeds in three stages corresponding signal durations of 10s, 30s, and 240s. Each stage focuses on learning the specific PPG feature target applicable to the duration. Subsequent stages freeze the learned parameter group and learn new targets thus preventing overwriting of useful features. RAE-PPG obtains best performance in 12/18 tasks while demonstrating the retain and extend abilities of pre-training.

Critical strengths
  • Practical importance: It is widely accepted that different health abnormalities manifest at different time scales in PPG. However, most modelling approaches do not account for this. RAE-PPG directly address this gap in the learning process, as such it presents an important contribution.
  • Evaluation: I commend the authors for providing confidence intervals for the results and performing statistical tests. Furthermore, experiments evaluating the RAE approach are comprehensive.
Critical weaknesses

I am willing to increase my score provided the following is adequately addressed:

  • Lack of comparison to SSL Baselines: RAE-PPG is compared to prior PPG-FMs which varying in architecture, training, size, etc. at the same time. Hence, the contribution of retain-and-extend pre-training to performance is unclear. To address this, RAE must be compared to SSL approaches such as SimCLR or TF-C using the same backbone and data used in RAE-PPG. This would demonstrate whether the performance gains stem specifically from multi-duration physiological feature supervision rather than the model architecture. I also recommend adding the parameter sizes during inference to the table.

  • Intermediate time durations: A central motivation of the paper is organizing learning around signal duration. However, the proposed architecture hardcodes duration into three static horizons (D={10,30,240} s\mathcal{D} = \{10, 30, 240\}\text{ s}). It seems that this design cannot natively ingest signals of arbitrary or intermediate lengths (e.g., 60 s or 120 s) at inference. How does RAE PPG plan to address this in the future? The limitations or future should expand on this.

Questions for authors

See weakness.

Reviewer_MVWxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RAE-PPG, a duration-grounded self-supervised pretraining framework for photoplethysmography (PPG) foundation models. Motivated by the observation that physiological signal features require different observation durations, the method progressively trains a single Transformer encoder using 10 s, 30 s, and 240 s inputs. It combines duration-specific physiological targets, a nested-read/protected-write parameter organization that freezes earlier parameter groups, and retained-target supervision intended to preserve previously learned information in longer-input representations.

The authors evaluate RAE-PPG on 18 downstream tasks across eight datasets, comparing against five existing PPG foundation models. RAE-PPG achieves the best observed performance on 12/18 tasks and ranks first or second on 17/18. Additional feature-decoding experiments and ablations investigate feature retention, extension, and reuse of previously learned parameters. The results support the feasibility of progressive duration-grounded pretraining, although the advantage of this training strategy over simpler joint alternatives remains unclear.

Critical strengths
  1. Physiologically motivated supervision with an interpretable mechanism for preserving prior learning.
  2. Extensive empirical evaluation and evidence of prior-stage knowledge reuse.
Critical weaknesses
  1. The necessity and advantage of progressive training are not convincingly established.

The paper motivates duration-appropriate physiological supervision, but it remains unclear why the associated targets must be learned sequentially with frozen parameter groups rather than jointly across multiple durations. The fully shared ablation retains the same sequential stage order and therefore does not address this question. Moreover, as acknowledged in Appendix D.1, this comparison simultaneously changes read access, parameter-update rules, and active model capacity.

A parameter- and compute-matched joint multi-duration baseline using the same physiological targets, as well as a direct long-duration training baseline, would be important to establish whether the proposed retain-and-extend mechanism offers benefits beyond the supervision design itself. Without these comparisons, it is difficult to attribute the reported improvements specifically to the proposed progressive training strategy. This also limits the assessment of novelty and significance given prior work on progressive networks and physiology-guided PPG representation learning.

  1. Feature retention across durations is incomplete, and the method exhibits a preservation–adaptation trade-off.

Although freezing earlier parameter groups guarantees unchanged computations for shorter inputs, this does not imply that their features remain equally accessible within longer-input representations. Appendix C.1 reports a mean decoding-gain reduction of 0.194 for 10 s features evaluated at 240 s, and 0.232 for 30 s features evaluated at 240 s, relative to their introduction-duration representations. Furthermore, downstream Retention is confirmed for only 2 of 8 applicable tasks.

The fully shared model with retained-target supervision also outperforms RAE-PPG on five of six 240 s downstream tasks, suggesting that the protection mechanism may constrain long-duration adaptation. These results do not invalidate the proposed method, but they qualify its central retention claim and leave the overall benefits of freezing versus adapting earlier representations insufficiently resolved.

Questions for authors
  1. Can the authors clarify why sequential duration-grounded pretraining is preferable to jointly training a single encoder on 10 s, 30 s, and 240 s inputs with the same 16 physiological targets? In particular, can they provide a parameter- and compute-matched comparison against joint multi-duration training or direct 240 s training with interval-aligned supervision? Such a comparison would help isolate whether the gains arise from the proposed progressive training mechanism or primarily from the duration-appropriate targets.
  2. The paper shows that shorter-input computations remain unchanged, but earlier-feature decodability decreases substantially at 240 s. What explains this degradation?
2 comments

Dear Reviewer,

As the ICLR 2027 discussion phase ends on November 18 at 11:59 PM AoE, please take a moment to review and respond to the authors' comments, even if your response is simply an acknowledgement. Your active participation in the discussion is greatly appreciated.

Thank you for your time and effort!

Thank you for the detailed rebuttal. I appreciate the authors' efforts to address my concerns.

The new 2×2 comparison of sequential versus interleaved training and protected versus shared parameters, together with Direct240, substantially addresses my request for controlled baselines. The additional global-pooling, token-resolution, and parameter-update experiments also clarify the degradation of earlier-feature accessibility and the preservation–adaptation trade-off.

My main remaining concern is the practical advantage and significance of progressive training. The results show task-dependent benefits rather than a consistent advantage over joint training. Moreover, while exact preservation of native-duration computations is established, its practical benefit relative to joint adaptation or checkpoint banking remains less clear. The comparisons also match update counts and input presentations rather than measured compute.

Given the concerns are partially address, and after carefully reading the paper again, I decided to keep my score.

Reviewer_ZSt6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose RAE-PPG, a PPG encoder pre-trained in three successive stages on 10 s, 30 s, and 240 s inputs. Self-supervision is regression of 16 handcrafted PPG features, each introduced at the first stage whose input is long enough to compute it. Each layer is split into duration-specific groups. A later stage reads the earlier groups but updates only its own, and earlier targets keep supervising the new group ("retain-and-extend"). Under linear probing on 18 tasks from 8 datasets, RAE-PPG has the best point score on 12 tasks against five PPG foundation models, including after the baselines are retrained on the same pre-training data.

Critical strengths
  1. The motivation makes sense. The vast majority of PPG FMs pre-train and evaluate on fixed-size inputs, when in reality many tasks depend on PPG signals at different durations.
  2. The benchmarking is broad and the evaluation protocol is clear and careful, including retraining all baselines on the same pre-training data.
Critical weaknesses
  1. One important concern is evaluation fairness. Each baseline is pre-trained at a single fixed length (10 s for PaPaGei and AnyPPG, 240 s for Pulse-PPG and SIGMA-PPG), and none is pre-trained across durations. At 10 s, where the comparison with AnyPPG is closest to fair, the two are about even, and RAE-PPG's 10 s representation is simply its stage-1 sub-network. One important baseline is therefore a simple GRU/LSTM over the sequence of fixed-length (e.g., 10 s) representations. It should be applied to the frozen baselines as well as to the authors' own stage-1 model. This would be a vital ablation to establish that the elaborate training scheme is necessary. Relatedly, a linear probe on the 16 handcrafted target features themselves would be a natural baseline, since the encoder is trained to regress them.
  2. I'm not convinced by the progressive setup, even though I agree with the variable-length motivation. When data at all three durations are already available, what's the motivation for progressive training? If it is about end performance, two ablations are important: 1) training a single model on all durations jointly, and 2) using the checkpoints at the stage boundaries. The appendix already hints at the second: from Tables 25–26, the stage checkpoints of the fully shared variant (B) beat RAE-PPG on 3 of 5 tasks at 30 s and 5 of 6 at 240 s. This also makes the retention study confusing: when the downstream input length is known in advance, simply using the boundary checkpoint guarantees perfect retention. If the motivation is instead a continual-learning style setup, a lot more justification is needed, since CL typically doesn't involve more pre-training.
Questions for authors

For the most important ones, please refer to the weakness section. Here are a few additional, less important, ones:

  1. How was the primary run chosen among the six pre-training seeds? It has the best mean rank of the six (1.44 vs. 1.56–1.72).
  2. Could the authors add paired tests against the strongest baseline for each task in Table 2? Several evaluation cohorts have only 6–8 participants.
1 comment

Dear Reviewer,

As the ICLR 2027 discussion phase ends on November 18 at 11:59 PM AoE, please take a moment to review and respond to the authors' comments, even if your response is simply an acknowledgement. Your active participation in the discussion is greatly appreciated.

Thank you for your time and effort!

Reviewer_4C8XOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RAE-PPG, a progressive pretraining framework that introduces physiological supervision across increasing PPG signal durations while preserving previously learned representations. Although the extensive experiments demonstrate promising downstream performance, several methodological and empirical concerns limit the paper's contributions.

Critical strengths
  1. The paper evaluates RAE-PPG on 18 tasks across eight datasets and compares it with five existing PPG foundation models. The proposed method achieves the best observed performance on 12 tasks, demonstrating promising transferability across different physiological applications.

  2. The authors provide feature decodability analyses, architectural ablations, and controlled comparisons of prior-stage knowledge reuse. These experiments offer useful insights into the behavior of progressive representation learning across different signal durations.

Critical weaknesses
  1. It remains unclear why representations learned from a limited set of handcrafted physiological targets should generalize effectively to diverse downstream tasks. Although the Feature-to-Task Bridge provides supporting evidence, only 18 of 33 examined relations are jointly supported. The authors should better explain the relationship between the predefined supervision targets and the generalization capabilities of the learned representations.

  2. The paper evaluates feature retention using pooling intervals aligned with shorter pretraining stages. Although this preserves the original feature analysis intervals, it may favor localized feature recovery rather than demonstrate information preservation in the complete longer duration representation. Moreover, downstream evaluation uses global mean pooling. The authors should justify this discrepancy and investigate whether the conclusions hold under consistent pooling strategies.

Questions for authors
  1. What evidence demonstrates that progressive duration-based pretraining is more effective than joint multi-duration training under comparable data, model capacity, and computational budgets? Could the authors clarify which improvements can be specifically attributed to the progressive training strategy?

  2. Why are shorter-stage pooling intervals used to evaluate continued feature decodability, whereas downstream tasks rely on global mean pooling? Can the authors clarify whether the retention conclusions remain valid when the same representation pooling strategy is applied?

  3. Given the substantial decline in earlier feature decodability at 240 s and the limited number of downstream tasks satisfying the Retention criterion, how do the authors reconcile these results with the central claim of effective knowledge preservation? What evidence supports the practical benefits of this preservation mechanism?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.