acceptodds
Under review as a conference paper at ICLR 2027

People over Segments: A Data-Axis Scaling Law for Person-Identity Contrastive PPG Pretraining

Abstract

Scaling data is the central lever behind foundation models, but a physiological corpus cannot be scaled like a web corpus: every additional segment must be recorded from a person, so growth is bounded not by what can be scraped but by who can be enrolled. A fixed segment budget therefore splits as K distinct people × M segments per person. Should a collector recruit more people, or record more from each? Both spend the same budget, yet for raw physiological waveforms the question is unsettled. We study this question for person-identity contrastive self-supervised learning on a real-world photoplethysmography (PPG) cohort of 114k people, in over 200 controlled pretraining runs. The number of distinct people seen in training is the stronger axis: segments per person help, but more slowly, and the most people-heavy allocation we sampled is best at every budget. More people do not merely lower the pretraining loss; they change what the representation encodes, separating held-out people more cleanly by identity, in step with better demographic prediction. A fitted scaling law makes the asymmetry quantitative: the people axis buys down its share of the error nearly twice as fast per decade as the segment axis, an ordering that holds across the fitting choices we test. Solving the law for its optimum yields a budget-adaptive acquisition rule: at small budgets it recruits more people than any allocation we sampled, and as the budget grows it shifts toward more segments per person. Training the rule's recommended points confirms it: each matches or beats the best allocation sampled at its budget. The message is simple: grow the number of distinct people before the volume of signal per person.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
wearable sensing, photoplethysmography (PPG), physiological time series, self-supervised learning, contrastive learning, scaling laws, data acquisition, foundation models

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_9otMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates how to allocate a fixed data budget between the number of participants and the number of PPG segments per participant for contrastive self-supervised pretraining. Using a cohort of 114K participants and over 200 pretraining runs, the authors show that increasing participant diversity yields greater benefits than collecting more segments per person. They further derive a two-axis scaling law and propose a budget-adaptive data acquisition strategy, validated through controlled experiments and downstream demographic prediction tasks.

Critical strengths

Practical research question.The paper addresses a relevant data allocation problem in physiological signal pretraining, examining the trade-off between recruiting more participants and collecting more segments per participant. Extensive empirical validation.The study conducts over 200 pretraining runs with controlled data budgets and proposes a two-axis scaling law. The experimental design provides quantitative evidence for data allocation decisions, although the broader generalizability remains unclear.

Critical weaknesses

W1. Limited generalizability. The proposed scaling law is primarily validated on a single PPG cohort using person-identity contrastive learning. Its applicability to other datasets, signal modalities, and self-supervised learning objectives remains unclear. W2. Potential evaluation bias. The main conclusions rely on demographic prediction tasks (age, BMI, and sex), which are naturally correlated with person identity. It is unclear whether the observed gains reflect general representation quality or alignment between the pretraining objective and downstream evaluations. Stronger validation on clinical tasks is needed. W3. Simplified data acquisition budget. The formulation (B=K\times M) assumes equal acquisition costs per segment, without accounting for the additional costs of recruiting new participants. This may limit the practical applicability of the proposed allocation strategy. W4. Insufficient validation across pretraining strategies. Although several SSL objectives are compared, the scaling analysis focuses on identity-contrastive pretraining. Additional experiments with alternative SSL objectives would help establish whether the observed scaling behavior is specific to the chosen training strategy.

Questions for authors

refer to the weakness

2 comments
AuthorsOpenReview ↗

Response to Reviewer 9otM

We thank Reviewer 9otM for the review and the clearly separated weaknesses, which we address in turn. All revisions are highlighted in blue in the revised PDF; those for this review are in Sections 1, 3, 5.4 and 7, and Appendices B.1, C.3, D.3 and E.8.

W1: Limited generalizability.

We agree that one pretraining cohort cannot establish a general law. The paper frames the result as a regularity of identity-contrastive PPG pretraining in this setting, and lists other cohorts, devices and modalities such as ECG as future work. Within that scope, the evidence now extends in three directions.

  • A new cohort. The within-person task (W2) uses a separate cohort of 1,034 people that we collected, independent of the pretraining data. The people-heavy allocation leads there at every budget.
  • Other backbones. The people-heavy end is also best for ResNet, ConvNeXt and ConvFormer backbones of matched size (Appendix D.3).
  • Other model sizes. The paper already shows the same ordering at three EfficientNet sizes, from 0.49M to 22.8M parameters (Appendix D.2).

Other objectives are covered in W4.

W2: Potential evaluation bias.

We agree that age, BMI and sex are tied to identity, so we added a target that is not: within-person changes in blood pressure on a separate cohort of 1,034 people, with each person's mean removed so that recognizing the person no longer helps the probe (General Response §1; Section 5.4, Appendix C.3).

The people-heavy allocation tracks these changes better at every budget for both SBP and DBP, by +0.019 to +0.055 in within-person correlation, and every 95% confidence interval of the gap excludes zero. The people-heavy allocation at 0.3M even matches or beats the segment-heavy allocation at 3M. The allocation ordering is therefore not limited to the demographic targets. On the clinical tasks the reviewer asks for, the existing labeled subsets (34 to 510 people) are too small to rank allocations, which is why we added a cohort with many blood-pressure readings per person.

W3: Simplified data acquisition budget.

This is a fair point. Sections 1 and 3 now state that B counts segments only, and Appendix E.8 adds a per-person recruitment cost to the allocator. A higher recruitment cost shifts the optimum toward fewer people with more segments each: slightly while recruitment is cheap relative to recording, and strongly once the two are comparable.

W4: Insufficient validation across pretraining strategies.

We trained the 1M diagonal with a masked autoencoder (Appendix B.1). The allocations tie, so the people advantage, the exponents and the allocator are specific to identity-contrastive pretraining, and the abstract and Discussion now state this scope. In our experiments, a people-heavy allocation is not worse beyond seed noise under either objective. Identity-contrastive training also has the highest clinical AUC of the seven objectives we compared (Table 5), so this scope covers the objective one would choose.

Summary

The within-person blood-pressure task on a separate cohort of 1,034 people shows that the allocation ordering holds beyond identity-linked traits. We hope these results address the reviewer's concerns and are glad to answer any further questions.

AuthorsOpenReview ↗

Follow-up to Reviewer 9otM

We thank Reviewer 9otM for continuing to follow the discussion; we noticed that the review was recently edited. As the discussion period ends today and the listed weaknesses refer to the original submission, we briefly summarize how the revision addresses each.

W1: limited generalizability. We agree that one pretraining cohort cannot establish a general law, and the paper frames the result as a regularity of identity-contrastive PPG pretraining in this setting. Within that scope, the ordering holds on a separate cohort of 1,034 people (Section 5.4), across four backbone families of matched size (Appendix D.3), and across three model sizes (Appendix D.2).

W2: potential evaluation bias. We added a target that is not tied to identity: within-person blood-pressure changes, with each person's mean removed so that recognizing the person no longer helps the probe. The people-heavy allocation leads at every budget (Section 5.4).

W3: simplified data acquisition budget. The paper now states that the budget counts segments only (Sections 1 and 3), and the allocator includes a per-person recruitment cost: a higher cost shifts the optimum toward fewer people with more segments each (Appendix E.8).

W4: insufficient validation across pretraining strategies. Under a masked autoencoder the allocations tie (Appendix B.1), so the paper now states its claim for identity-contrastive pretraining only. This scope covers the objective one would choose: identity-contrastive training gives the highest clinical AUC of the seven objectives we compared (Table 5).

We hope this resolves the reviewer's concerns. If any remain, we would be glad to discuss them before the discussion period ends; if not, we would be grateful if the assessment could reflect the revision.

Reviewer_mc7UOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses a practically relevant data collection problem. Its main claim is that increasing participant diversity is generally a more effective use of a fixed PPG data budget than increasing the amount of signal data collected per participant, particularly when the budget is limited. Specifically, the authors conduct person-identity contrastive self-supervised pretraining on a large photoplethysmography (PPG) cohort, performing more than 200 controlled pretraining runs. They compare different allocations between the number of individuals and the number of segments collected per individual while keeping the total number of segments fixed. The authors further examine identity-related properties of the learned representations and fit a two-axis scaling law to quantify the relative contributions of the two data dimensions.

Critical strengths
  1. The research motivation is reasonable and relevant, and the findings may help inform future dataset construction strategies.
  2. The experimental design is reasonable. The authors conduct experiments across different data budgets, with repeated random seeds and additional evaluations using models of different capacities. The experimental results are consistent with the study's motivation, and the paper is generally well written and clearly presented.
Critical weaknesses
  1. The scaling law relies on a specific functional form. The fitted model uses an additive, separable power law with distinct exponents for the number of individuals and the number of segments per individual. However, the practical stability of the recommended allocations warrants further investigation.
  2. The cost model does not capture all practical data collection costs. The fixed budget is defined primarily by the total number of segments. Although the paper discusses these practical considerations, the proposed allocation rule does not directly optimize a cost model that incorporates them.
  3. The empirical variability and effect sizes differ across budget levels. The people-heavy allocation performs best at every tested budget, but the performance gap narrows as the budget increases. This does not invalidate the overall trend, but it suggests that the practical value of changing the allocation depends on the available budget and should be characterized accordingly.
Questions for authors
  1. How stable are the recommended allocations under alternative functional forms, different assumptions about the error floor, and uncertainty in the fitted exponents? Could the authors quantify the uncertainty in the recommended allocations rather than reporting only a single optimal value?
  2. How would the recommended allocation change if the budget explicitly accounted for participant recruitment and retention costs, rather than treating the total number of segments as the primary cost? A sensitivity analysis under different relative recruitment costs would help clarify this issue.
  3. Since the performance gap between different allocations narrows as the total budget increases, is there a principled way to identify the budget range in which recruiting more participants yields greater benefits than collecting more segments per participant?
2 comments
AuthorsOpenReview ↗

Response to Reviewer mc7U

We thank Reviewer mc7U for the review and for focusing on how the allocation rule would be used in practice. The reviewer's questions led us to report the rule together with its uncertainty and its dependence on budget and cost. All revisions are highlighted in blue in the revised PDF; those for this review are in Sections 5.2, 6.1 and 6.3, Table 2, and Appendix E.8. Each weakness pairs with one of the reviewer's questions, so we answer them together.

W1 / Q1: Stability of the recommended allocation

The ordering of allocations is measured directly and does not depend on the functional form; the exact recommended headcount does. We recomputed K* while changing one fitting choice at a time: the edge policy, a fixed error floor, the interaction model, and seed resampling (Appendix E.8). At 0.3M, K* ranges from 3.9k to 25.5k, so Table 2 now reports its range and we read it as a scale rather than a precise value. Two conclusions hold in every variant: at the smallest budget the recommendation is more people-heavy than any allocation we sampled, and K* grows with budget. The trained recommendation at 0.3M also beats every sampled allocation, by +0.036 over the best one.

W2 / Q2: Recruitment and retention cost

We added a fixed cost per person, covering recruitment and retention, so that C=cKK+cMKMC = c_K K + c_M KM, and re-solved the allocator over a range of cost ratios (Appendix E.8). A higher per-person cost moves the optimum toward fewer people with more segments each. The shift is small while one person costs much less than recording the recommended segments: at 10M, a person costing as much as 10 segments lowers K* by 4%. It is large once the two are comparable: at 1M, a person costing 100 segments lowers K* by 65%.

W3 / Q3: Where recruiting more people pays off

On the composite, recruiting more people pays off most at small budgets and matters little at the largest one (Table 10). At 0.3M and 1M, the people-heavy allocation leads the most segment-heavy one by 0.100 and 0.130 in composite, far outside seed noise, while at 10M all allocations are close. The fitted law marks the boundary with M*(B), the segments per person at which the two axes return equally: recruiting more people pays off while each person contributes more than M*(B). At large budgets the composite is nearly flat around this point: at 10M, the recommended allocation comes within 0.004 of the best grid arm with 72% fewer people, so per-person cost can decide. On within-person blood pressure in a separate cohort of 1,034 people (General Response §1), recruitment still pays at 10M: the people-heavy allocation leads by +0.019 (SBP) and +0.025 (DBP) in within-person correlation, with both confidence intervals excluding zero. Section 5.2 now describes the value of the allocation budget by budget.

Summary

The measured ordering of allocations is stable, and the controls in the General Response show that it is not an artifact of the setup. The exact K* is not stable, so we now report its range. We hope these results address the reviewer's concerns and are glad to answer any further questions.

AuthorsOpenReview ↗

Follow-up to Reviewer mc7U

We thank Reviewer mc7U again for the review. As the discussion period ends today, we briefly summarize how the revision answers each weakness and its paired question, and what a practitioner can take from the allocation rule.

W1 / Q1: stability of the recommended allocation. We agree that the exact recommended headcount is not stable, so Table 2 now reports K* as a range across fitting choices, and the paper reads it as a scale rather than a precise value. The direction is stable: at the smallest budget every fitting variant recommends more people than any allocation we sampled, the trained recommendation beats every sampled allocation, and K* grows with budget in every variant. The measured ordering of allocations does not depend on the fit.

W2 / Q2: recruitment and retention cost. The allocator now includes a fixed cost per person (Appendix E.8). A higher per-person cost shifts the optimum toward fewer people with more segments each: slightly while one person costs much less than recording the recommended segments, and strongly once the two are comparable.

W3 / Q3: where recruiting more people pays off. On the composite, it pays off most at small budgets, where the people-heavy allocation beats the segment-heavy one given about three times the budget. At the largest budget all allocations are close, so per-person cost can decide (Section 5.2). On within-person blood pressure, recruiting more people still pays at the largest budget (Section 5.4).

Practical takeaway. The rule gives a collector a direction rather than a precise headcount: at small budgets, favor people over segments per person; as the budget grows, shift toward more segments per person; and once the allocations converge, let per-person cost decide.

We would be glad to clarify any of these before the discussion closes, and hope the revision can be taken into account in the final assessment.

Reviewer_64S5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors study how a fixed pre-training budget of B = K × M PPG segments should be split between K people and M segments per person under a participant-contrastive objective. They run 200+ pre-training runs on a 114k-person smartwatch cohort and score each run with a demographic-prediction composite (age, BMI, sex). The most people-heavy allocation wins at every budget. A fitted scaling law yields a budget-dependent allocator, whose recommended splits the authors train and validate.

Critical strengths
  1. The motivation is clear and interesting. Selecting the pre-training corpus is an extremely important aspect, especially considering how expensive large-scale data collection and pre-training can be.
  2. The fixed-budget grid is clean and the evidence is unusually convincing within its scope: people-heavy allocations win at every budget and every model size, independent of the fitted law.
  3. The limitations section is honest and clear.
Critical weaknesses
  1. My main concern is on the y-axis. The composite measures stable, person-level traits on per-person averaged embeddings. That is the same axis the objective optimizes, so the composite can't capture what additional segments per person contribute. The authors argue that more segments buy longitudinal coverage (rest, exertion, sleep), but no task that depends on within-person state is evaluated (even BP is a single cuff reading per person and is stashed to the appendix). Since evaluation is linear probing, extending it is cheap, and there are many mature benchmarks for this. The common tasks used by PPG FM papers such as PaPaGei (Pillai et al., 2025), Pulse-PPG (Saha et al., 2025), and AnyPPG (Nie et al., 2025) span tasks like HR, RR, AF, and BP, and can be evaluated at the patient or segment level. In EEG, Bomatter & Gouk (2025) found the answer to this question is task-dependent, so I consider this evaluation necessary for acceptance.
  2. The conclusion may be too specific to the training objective. The title and limitations scope the claim to identity-contrastive pre-training, which I can accept, but the abstract and conclusion still read as a general rule for collecting pre-training data. Under this loss, headcount directly sets the coverage of negative pairs, so it is not too surprising that it dominates. The LSM study the authors cite (Narayanswamy et al., 2025) found the opposite under masked reconstruction. I'd like to see the ordering hold for at least one non-identity objective (e.g., the MAE in Table 3) along a single diagonal, or the abstract toned down to match the title.
  3. A small architecture ablation would be helpful, though less important.

Citations:

  1. Pillai, A., Spathis, D., Kawsar, F., & Malekzadeh, M. (2025). PaPaGei: Open foundation models for optical physiological signals. arXiv preprint arXiv:2410.20542.
  2. Saha, M., Xu, M. A., Mao, W., Neupane, S., Rehg, J. M., & Kumar, S. (2025). Pulse-PPG: An open-source field-trained PPG foundation model for wearable applications across lab and field settings. arXiv preprint arXiv:2502.01108.
  3. Nie, G., Fang, X., Tang, G., Xiao, Y., Li, J., Liu, B., Li, H., & Hong, S. (2025). AnyPPG: An ECG-guided PPG foundation model trained on over 100,000 hours of recordings for holistic health profiling. arXiv preprint arXiv:2511.01747.
  4. Bomatter, P., & Gouk, H. (2025). Is limited participant diversity impeding EEG-based machine learning? arXiv preprint arXiv:2503.13497.
Questions for authors

The main questions are above at the weaknesses section. Here are some less important ones.

  1. How are the M segments per person sampled, and are segment-heavy arms limited to participants with at least M segments? If so, do the demographics differ across arms?
  2. Does the small-budget people advantage hold with identity-balanced batches or matched step counts?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 64S5

We thank Reviewer 64S5 for the detailed review. The reviewer's point about the y-axis, and the pointer to Bomatter & Gouk, shaped the main addition of this revision. All revisions are highlighted in blue in the revised PDF; those for this review are in Sections 2, 5.3 and 5.4, and Appendices A.4, A.5, B.1, C.3 and D.3.

W1: The composite can't capture what additional segments per person contribute.

We agree, and added a task that can, now in the main text (Section 5.4, Figure 4, Appendix C.3). We chose blood pressure because it gives many labeled readings per person, enough to rank allocations on within-person changes. The task uses a separate cohort of 1,034 people with 70,371 cuff readings, a median of 67 per person over about two weeks, none of them in any pretraining pool. The probe works on individual readings rather than per-person averages. Each person's mean is removed from both the embeddings and the readings, so what remains is how a person's state changes between readings.

On this target, the people-heavy allocation leads at every budget for both SBP and DBP, by +0.019 to +0.055 in within-person correlation, and every 95% confidence interval of the gap excludes zero. Even when the target varies within a person, extra segments per person do not let segment-heavy allocations catch up. The people-heavy allocation at 0.3M even matches or beats the segment-heavy allocation at 3M, ten times the budget. Bomatter & Gouk found the answer task-dependent in EEG, and Related Work now cites them. In our setting the ordering is the same for trait and within-person targets, and what removes it is the objective (W2). We did not add segment-level HR/RR benchmarks in this revision; among the clinical labels we have, AFib covers only 34 people, too few to rank allocations.

W2: The conclusion may be too specific to the training objective.

We did both things the reviewer suggested. Along the 1M diagonal, a masked autoencoder shows no allocation effect: the five arms score 0.409–0.416, within seed noise (Appendix B.1). This is consistent with LSM's reconstruction result, so the objective can explain why the two studies differ. The abstract and Discussion now state the claim for identity-contrastive pretraining. Since the composite may favor identity objectives, we also compared the two on the clinical endpoints, where identity-contrastive training is again stronger (overall AUC 0.902 against 0.874; Table 5).

W3: A small architecture ablation would be helpful.

We trained ResNet, ConvNeXt and ConvFormer backbones of matched size on the 1M diagonal (Appendix D.3). On every backbone the people-heavy end wins and the allocations keep the same order. The size of the gap varies: ResNet (+0.134) and ConvFormer (+0.171) match or exceed EfficientNet (+0.130), while ConvNeXt's is smaller (+0.036). Since the training recipe was not re-tuned for the new backbones, we claim the direction of the effect for them, not its size.

Q1: Segment sampling and eligibility

Segments are drawn at random from each person's whole recording, and a contiguous-block sampler gives nearly the same results (Appendix A.5). Yes, an arm enrolls only people with at least M segments, so segment-heavy arms on the fixed-budget diagonal draw from people who wear the device more, and their demographics may differ. This affects only the diagonal. The orthogonal sweeps hold the threshold fixed: the K-sweep draws every arm from people with at least 250 segments, and the M-sweep uses the same 10,000 people at every M. The per-axis returns they measure, including the slow gain along M, therefore do not depend on who is enrolled.

Q2: Identity-balanced batches and matched steps

Yes on both. We retrained the whole grid with every batch fixed at 128 people × 2 segments. The people-heavy arm still leads at every budget, including 0.3M (+0.092, originally +0.100) and 1M (+0.116, originally +0.130) (Appendix A.4). Training steps were already matched within each budget, and the balanced runs keep this.

Summary

The reviewer described a within-person evaluation as necessary for acceptance. On within-person blood pressure in a separate cohort of 1,034 people, the people-heavy allocation leads at every budget, so the ordering is not specific to stable traits. Under a masked autoencoder the allocation does not matter, and the paper now states its claim for identity-contrastive pretraining. We hope these results address the reviewer's concerns and are glad to answer any further questions.

I thank the authors for the response. I think W2-3 are fully addressed.

However, could the author maybe show some effect on one or two public HR/RR benchmarks? I think that would make the paper even more convincing. WESAD, for example, could be a strong candidate for HR.

AuthorsOpenReview ↗

Response to Reviewer 64S5

We thank Reviewer 64S5 for confirming that W2 and W3 are fully addressed and for suggesting public HR/RR benchmarks. We evaluated frozen representations on PPG-DaLiA for heart-rate variation, with WESAD stress classification as a supplementary task.

Evaluation protocol

Both public datasets use an Empatica E4 wrist device, different from our pretraining device. For each dataset, we evaluated all 20 pretraining configurations with three seeds each, using 15 participants, 15-fold leave-one-subject-out testing, and inner three-fold subject-level cross-validation. Standardization is fitted only on the corresponding training fold. In both tables, entries are mean ± sample SD across three seeds, not confidence intervals. M denotes pretraining segments per person; smaller M means more people at a fixed segment budget.

PPG-DaLiA: within-person heart-rate tracking

We used 64,522 PPG windows, each containing 30 seconds centered on an official 8-second HR-label interval. A ridge probe predicts HR deviations from person-centered embeddings. For the held-out participant, the embedding mean uses the full input recording, including future windows. The held-out participant's HR mean defines evaluation targets only and never enters probe fitting or hyperparameter selection. We report pooled Pearson r between person-centered predictions and targets. This measures retrospective within-person HR tracking, not absolute-HR accuracy or real-time estimation.

Pretraining budgetM=125M=250M=500M=1000M=2000
0.3M.750 ± .002.760 ± .009.742 ± .006.747 ± .015.756 ± .014
1M.761 ± .016.742 ± .011.760 ± .011.755 ± .014.753 ± .019
3M.762 ± .010.759 ± .017.764 ± .006.760 ± .002.744 ± .023
10M.767 ± .008.761 ± .010.781 ± .020.761 ± .014.773 ± .012

WESAD: supplementary stress classification

A class-balanced logistic probe distinguishes stress from baseline/amusement on 3,175 valid 30-second windows. The metric is mean held-out-participant AUROC, with equal weight per participant. This is a stress result, not an additional HR/RR evaluation.

Pretraining budgetM=125M=250M=500M=1000M=2000
0.3M.9126 ± .0133.9209 ± .0108.9252 ± .0066.9251 ± .0052.9260 ± .0103
1M.9306 ± .0085.9092 ± .0153.9216 ± .0083.9294 ± .0052.9299 ± .0114
3M.9323 ± .0093.9222 ± .0101.9276 ± .0092.9305 ± .0097.9328 ± .0105
10M.9303 ± .0068.9231 ± .0045.9318 ± .0075.9286 ± .0086.9211 ± .0104

What these results establish

The frozen representations contain task-relevant information on both public datasets, but neither task shows a consistent people-heavy advantage. All eight paired 95% confidence intervals for the people-heavy minus segment-heavy difference, across four budgets and two tasks, include zero, using 50,000 joint seed-and-subject bootstrap replicates. This does not establish equivalence between allocations.

Differences in hardware, recording conditions, and downstream targets may affect the allocation ranking, although these experiments do not isolate their effects. The public evaluations extend the evidence beyond our acquisition setting and distinguish representation transfer from transfer of the allocation ranking. The paper's central contribution is a controlled quantification of the returns to adding people versus segments at a fixed budget. The demographic experiments support the people-heavy ordering, and the separate 1,034-person within-person BP experiment shows that this finding is not restricted to stable traits: people-heavy allocation leads at every budget for both SBP and DBP, with all endpoint-gap 95% confidence intervals excluding zero.

Reviewer_UTPhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper explores whether, when there is a fixed PPG data budget, it is better to record data from more people or to record more data from each subject or person by using 114k people, in over 200 controlled pretraining runs. Using more people always wins, including on blood-pressure changes within a person. However, adding people helps about twice as much as adding recordings. So the advice is to add people first and add recordings per person later, as the budget grows.

Critical strengths
  • Carefully controlled experiments. The paper combines fixed-budget allocation grids with sweeps along each axis. It uses one fixed evaluation set, nested sampling and paired seeds, so only the allocation changes between runs.
  • A consistent main finding. The people-heavy split wins at all four budgets and three model sizes. It also agrees with validation loss and with identity separation on held-out people, which makes it the paper’s most reliable result.
Critical weaknesses
  • The setup favours the people axis by construction. With the identity-contrastive objective, the number of people sets how many distinct negatives each batch contains, so segment-heavy runs face a weaker training task. The main metric (age, BMI, sex) measures traits that don’t change within a person. Within-person tasks, where more recordings per person should help most, are too small to test (34 to 200 people). The conclusion therefore applies to person-level traits under this objective, not to PPG pretraining in general.
  • The scaling law and allocation rule are fragile. The people exponent ranges from 0.27 to 0.48 depending on how noisy points are handled. The segment exponent drops to about 0.05 under the better-fitting model. Beyond the smallest budget, the recommended allocation differs from the best tested split only within seed noise. The cost model also ignores per-person recruitment costs, so the rule is of limited use for real data collection.
Questions for authors
  • How are positive pairs and batches constructed? Are segments sampled uniformly over a person’s segments, so that low-KK arms have many same-person segments per batch? Have you run, or could you run, a control that fixes the number of distinct identities per batch (or segments per identity per batch) across allocations? If K-dominance survives that control, my first weakness is largely resolved.
  • For a given MM, are a person’s segments drawn at random across their whole recording span, or as a contiguous block? At M=125M=125, random sampling already gives longitudinal coverage, which would explain why the MM axis looks redundant.
  • How is the linear probe trained and evaluated within the 5,000 held-out people: a train/test split or k-fold? How is the sex target handled as an MAE regression?
  • Do you have any within-person or state-dependent downstream task with enough labels to test whether the MM axis matters more there? Examples are per-segment heart-rate or HRV regression, activity/sleep state, or BP changes within a subject. Even one adequately powered within-person task would show whether the people-first conclusion holds beyond person-level traits.
  • How were the ± uncertainties on α and β computed? Given the interaction model’s much better BIC and its β ≈ 0.05, why is the separable model the one used for the headline exponents and for the allocator?
  • Re-solve the allocator under a two-term cost cKK+cMKMc_K K + c_M K M with realistic cost ratios, and test identity-invariant or reconstruction objectives on the same grid to bound the scope of the claim.
3 comments
AuthorsOpenReview ↗

Response to Reviewer UTPh (1/2)

We thank Reviewer UTPh for the careful review and for proposing a concrete control for batch composition. We ran it on the full grid, together with the other analyses the reviewer asked for. All revisions are highlighted in blue in the revised PDF; those for this review are in Sections 5.2, 5.4 and 6.1, Table 2, and Appendices A.4, A.5, B.1, C.3 and E.8.

W1: The setup favors the people axis by construction.

We tested the three parts of this argument separately.

  • Batch composition. With identity-balanced batches, the people advantage changes by at most 0.014 at any budget (Q1).
  • Targets. On within-person blood-pressure changes in a separate cohort of 1,034 people, the people-heavy allocation leads at every budget for both SBP and DBP (Q4).
  • Objective. Under a masked autoencoder, the allocations tie (Q6).

The advantage therefore needs the identity objective, as the reviewer suspected, but it does not come from batch make-up and is not limited to stable traits. The paper now states its claim for identity-contrastive pretraining.

W2: The scaling law and allocation rule are fragile.

We agree that the exact values are fragile, and have revised the claims accordingly. The paper now claims the ordering α > β rather than particular exponent values. Every variant the reviewer cites keeps this ordering: α stays above β under all three edge policies, and the better-fitting interaction model, where β drops to about 0.05, widens the gap rather than closing it. K* is reported with its range across fitting choices and read as a scale (Table 2): at 0.3M it ranges from 3.9k to 25.5k, and every variant recommends more people than the grid's most people-heavy arm (2.4k). The forward validation is now described as the reviewer read it: the recommendation beats the best grid arm at 0.3M (+0.036), matches it within seed noise at 1M and 3M, and is slightly below it at 10M. The measured ordering of allocations, which is the paper's main result, does not depend on the fit.

Q1: Batch construction, and a control on identities per batch

K-dominance survives this control: with every batch fixed at 128 people × 2 segments, the people-heavy end still wins at every budget by a similar margin (Section 5.2, Appendix A.4). A segment's positive is another segment of the same person, other people's segments in the batch are its negatives, and any further segments of the same person are masked out of the InfoNCE denominator, so a person is never contrasted against themselves (Section 3.1). Segments are drawn uniformly, so low-K arms do repeat people within a batch, as the reviewer suspected. We retrained the full fixed-budget grid with every batch holding 128 people × 2 segments, keeping batch size and training steps fixed. The people advantage per budget:

People advantage (composite)0.3M1M3M10M
Original sampler+0.100+0.130+0.043+0.012
Identity-balanced sampler+0.092+0.116+0.042+0.013

If anything, this control works against the people-heavy arms, since it lowers the number of distinct people in their batches.

Q2: Random or contiguous segments

They are drawn at random across each person's whole recording. We retrained the 1M diagonal and the M-sweep with contiguous blocks. The gain from M = 125 to 1000 rises only slightly (+0.046 against +0.038), the 1M people advantage stays large (+0.114 against +0.130), and the allocations keep their order (Appendix A.5).

Q3: Probe protocol and the sex target

The probe is a linear model on each person's mean embedding, and every run is scored on the same 5,000 held-out people, who are outside every pretraining pool. Within these 5,000 people, the probe is fit and scored by 3-fold cross-validation across people: each person's prediction comes from a probe fit on the other two folds, so no person is scored by a probe that saw their own labels (Appendix A.2). Sex is fit by the same ridge probe as age and BMI, and its MAE enters the composite as a skill relative to a constant predictor (Equation 3), so the three targets share one scale.

AuthorsOpenReview ↗

Response to Reviewer UTPh (2/2)

Q4: A within-person task with enough labels

Yes. We added the third example the reviewer lists, blood-pressure changes within a subject, on a separate cohort of 1,034 people with 70,371 cuff readings, a median of 67 per person over about two weeks (Section 5.4, Figure 4, Appendix C.3). The cohort is separate from the paper's 200-person blood-pressure subset, and none of its people is in any pretraining pool. Most of the variance in blood pressure lies between people, so a probe scored on raw readings would mostly be scored on each person's average level, another stable trait. We therefore remove each person's mean from both the embeddings and the readings, so the probe is scored only on within-person changes.

With 70,371 readings the task has enough power to separate allocations, and the M axis does not matter more there. The people-heavy arm leads at every budget for both SBP and DBP, by +0.019 to +0.055 in within-person correlation, and every 95% confidence interval of the gap excludes zero (per-budget values in the General Response). From 1M up, the five allocations rank in order of M. The people-heavy arm at 0.3M matches or beats the segment-heavy arm at 3M, and at 10M the gap is still about 40% of its 1M size, against under a tenth on the composite. The probe tracks only part of the within-person variation, so the task tests how allocations rank, not clinical accuracy.

Q5: The ± on α and β, and the separable model

They are standard errors from the fit covariance, which treat seed spread as the only noise and are therefore too small. Section 6.1 now also gives errors scaled to the fit residuals (±0.050 for α, ±0.115 for β). We keep the separable model because it gives one rate per axis, which is the comparison the paper makes. The interaction model's extra term depends on K and M only through K·M, so it is the same for every allocation at a fixed budget. With it, β drops further and the recommended headcount rises, so the separable fit is the conservative choice for both the exponents and the allocator.

Q6: A two-term cost, and a non-identity objective

Since recruitment costs differ widely between studies, we re-solved the allocator under C=cKK+cMKMC = c_K K + c_M KM over a range of cost ratios (Appendix E.8). A higher recruitment cost moves the optimum toward fewer people with more segments each: little while one person costs much less than recording the recommended segments, and strongly once the two are comparable. For the objective, a masked autoencoder on the 1M diagonal shows no allocation effect: all five arms score 0.409–0.416, within seed noise (Appendix B.1). This bounds the claim to identity-contrastive pretraining.

Summary

The reviewer noted that if K-dominance survived a batch-composition control, the first concern would be largely resolved. It survives that control at every budget, and it also holds on within-person blood pressure in a separate cohort of 1,034 people. It depends on the identity objective, which the paper now states as its scope, and the exponents and recommended allocation are now reported with their uncertainty. We hope these results address the reviewer's concerns and are glad to answer any further questions.

AuthorsOpenReview ↗

Follow-up to Reviewer UTPh

We thank Reviewer UTPh for reading our response and for updating the summary to include the within-person blood-pressure result. Since the listed weaknesses refer to the original submission, we briefly summarize how the revision addresses each.

W1: the setup favors the people axis. The people advantage does not come from batch make-up: with the same number of people in every batch, it barely changes at any budget (Appendix A.4). It is also not limited to stable traits: on within-person blood pressure in a separate cohort, the people-heavy allocation leads at every budget (Section 5.4). It does depend on the identity objective, as the reviewer suspected, so the paper now states its claim for identity-contrastive pretraining only (Appendix B.1).

W2: the law and allocator are fragile. We agree that the exact values are fragile, and the revision claims less accordingly. The paper now claims only the ordering α > β, which holds across the edge treatments and model forms we test, rather than particular exponent values. It reports K* as a range rather than a precise headcount (Table 2), and it describes the forward validation as measured: the recommendation beats the best tested split at the smallest budget and stays close to it elsewhere. The paper's main result, the measured ordering of allocations, does not depend on the fit. This ordering and the allocator's shift toward more recordings per person at larger budgets are consistent: at large budgets the people-heavy allocation still leads, but by little (+0.012 at 10M), so the shift reflects diminishing returns to headcount, not a reversal.

We hope this resolves the reviewer's concerns. If any remain, we would be glad to discuss them before the discussion period ends; if not, we would be grateful if the assessment could reflect the revision.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for their constructive reviews. Reviewers found the allocation question practically important (64S5, mc7U, 9otM), the fixed-budget design carefully controlled (UTPh, 64S5, mc7U), and the people-heavy result consistent across budgets and model sizes (UTPh, 64S5). All revisions are highlighted in blue in the revised PDF.

The reviews raised two shared concerns. The first is whether the people advantage is built into the setup, through the demographic targets, the batch composition or the identity objective. The second is how far the fitted allocation rule can be trusted, including once recruitment costs are counted. We ran the main controls the reviewers proposed, including a retraining of the full grid with identity-balanced batches, and added a separate cohort of 1,034 people for a within-person task. The people advantage survives the batch, sampling and backbone controls, and it extends from stable traits to within-person changes in blood pressure. The objective experiment ties it to the identity objective, which can account for why reconstruction-based work such as LSM reached a different conclusion.

Concern (reviewers)New evidenceResultWhere
Identity-linked targets (UTPh, 64S5, 9otM)Within-person BP, new cohort of 1,034 peoplePeople-heavy leads at every budgetSec. 5.4, App. C.3
Batch composition (UTPh, 64S5)Full grid with 128 people × 2 segments per batchGap moves by ≤ 0.014App. A.4
Segment sampling (UTPh, 64S5)Contiguous-block samplerSame orderingApp. A.5
Backbone (64S5)ResNet, ConvNeXt, ConvFormerPeople-heavy best on all fourApp. D.3
Objective (UTPh, 64S5, 9otM)Masked autoencoderAllocations tie; claim scopedApp. B.1
Allocator (UTPh, mc7U, 9otM)K* range across fits; recruitment costDirection holds at small budgetsTable 2, App. E.8

1. The people advantage holds on within-person blood pressure (new cohort, 1,034 people)

Age, BMI and sex are stable within a person, so they cannot show what extra segments per person add. We added a within-person task on a separate cohort: 1,034 people with 70,371 cuff readings, a median of 67 per person over two weeks, none of them in any pretraining pool. Each person's mean is removed from both the embeddings and the readings, so the probe is scored only on how a person's blood pressure changes. On this task, where extra segments per person would be expected to help most, the people-heavy allocation leads at every budget for both SBP and DBP, and every 95% CI of the gap excludes zero:

People-heavy minus segment-heavy0.3M1M3M10M
Within-person r, SBP+0.027+0.050+0.051+0.019
Within-person r, DBP+0.035+0.055+0.051+0.025
Demographic composite+0.100+0.130+0.043+0.012

The allocation is worth more than a tenfold budget: the people-heavy allocation at 0.3M matches or beats the segment-heavy allocation at 3M. The advantage also lasts longer than on the composite: at 10M it is still about 40% of its 1M size.

2. Batch composition, segment sampling and backbone do not explain it

With every batch in every arm fixed at 128 people × 2 segments, the people advantage stays at +0.092, +0.116, +0.042 and +0.013 across the four budgets (originally +0.100, +0.130, +0.043, +0.012). Contiguous segment sampling keeps the order of the allocations, and the people-heavy end is best on all four backbone families.

3. It requires the identity objective; the claim is now scoped accordingly

To test whether the advantage depends on the objective, we retrained the five 1M allocations with a masked autoencoder. The allocations tie (gap −0.002, against +0.130), consistent with LSM. The paper now states its claim for identity-contrastive pretraining, which also has the highest clinical AUC of the seven objectives we compared (Table 5). Under the masked autoencoder, the people-heavy allocation scores on par with the others, so favoring people does not hurt there either.

4. The measured ordering does not depend on the fit; K* is now given as a range

The measured result, that the most people-heavy allocation is best at every budget, does not depend on the fitted law. The exact K* does, so Table 2 now reports its range across fitting choices and we read it as a scale. At 0.3M every variant recommends more people than any allocation we sampled, and the trained recommendation beats them all. The allocator now also includes a per-person recruitment cost.

We thank the reviewers again and are glad to discuss any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.