acceptodds
Under review as a conference paper at ICLR 2027

Longer Records, Broader Invariance: The Hidden Scaling Problem in Longitudinal Contrastive Learning

Abstract

Longitudinal data are valuable because people change. Yet the objectives used to learn from these data can inadvertently erase that change. In person-level contrastive learning, observations from the same person are treated as positives; as records grow, those positives can span increasingly distant—and increasingly different—behavioral states. More history can therefore produce not only more data, but broader invariance. We show that this distinction is fundamental. We separate record span, how much history the learner sees, from supervision span, how far across that history positive-pair supervision reaches. Across in-home sensing records spanning up to 2.7 years, broader supervision systematically suppresses recoverable changing-state information, even when the available history is held fixed. At the broadest span, less than 10% of the information recoverable from an untrained encoder remains. Yet keeping positives local is not sufficient: as records grow, even distant states that are never paired become increasingly similar. Explicitly contrasting other observations from the same person reverses this loss without shortening the record, revealing a second route by which longitudinal scale can broaden invariance. Finally, we prospectively reproduce the supervision-span effect in 199 GLOBEM participants. Longitudinal scale therefore presents a choice: more history need not mean more invariance. By controlling what is held invariant as records grow, we can preserve the change that made the longitudinal data valuable in the first place.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
self-supervised learning, contrastive learning, longitudinal data, representation learning, temporal representation learning, invariance, time series

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FvwjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper deals with longitudinal person-level contrastive learning. They discuss the problem that over time, samples from the same person should not necessarily be treated as positive paris, as they may change significantly. They show empirical evidence in 2 in-home sensing datasets that information is lost because of these longitudinal changes. They propose to contrast other observations from the same person (i.e. controlling what is considered invariant).

Critical strengths
  1. The paper is mostly well presented with a logical flow.
  2. The central idea is clearly explained.
Critical weaknesses
  1. I understand the problem you are tackling, but I am unclear whether this is really a relevant/important/existing problem. Do other works really do longitudinal person-level contrastive learning over long time periods? And does this cause models to underperform? In any case, you should definitely expand on this in the introduction and further discuss related works.
  2. I am unclear about what the task you are really trying to do is. Why is it of interest to have as a primary metric the information shared between two samples? I feel like this should be coupled to downstream tasks to show actual impact of the phenomena you observe/method you propose.
  3. Experiments are lacking in breadth. Only two small datasets are studied.

[Not factored into score] 4. The appendix is very long and contains 50+ tables. Can you summarise what the appendix shows? Are all results/explanations in the appendix necessary to the paper? 5. The paper writing sounds (to me) extremely AI-generated. This may frustrate future readers.

Questions for authors

See weaknesses

Reviewer_ioXWOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper tries to address whether person-level contrastive learning on longitudinal data loses information about a person's changing state as records get longer. The authors separate record span (how much history the learner sees) from supervision span (how far apart positive pairs can be), and measure changing-state information with a conditional predictive quantity (ΔI). Using in-home sensing data from DETECT (57 single-resident participants and 44 households, 48 daily features) and an MLP encoder of about 40k parameters, they find that broader supervision reduces ΔI, that longer records reduce ΔI even when supervision stays local, and that contrasting other periods of the same person recovers the loss. They also reproduce the supervision-span effect prospectively on 199 GLOBEM participants.

Critical strengths

I think the question itself is meaningful. Separating record span from supervision span (pp. 1-2) points to a real confound in longitudinal self-supervised learning: as records grow, same-person sampling quietly widens the invariance the model is asked to learn. The paper also shows some signal for this. The supervision-span effect appears in both DETECT cohorts and is reproduced on GLOBEM with a feature rule frozen in advance (5/5 seeds, p. 38), and the authors run a large number of controls and variants. For these reasons I think the work could be considered for a workshop.

Critical weaknesses
  1. I think the experimental scale cannot support the conclusion. The paper is framed around the scaling of longitudinal and wearable foundation models (p. 1), but all experiments use about a hundred units with 48 daily features and a three-layer MLP of about 40k parameters (pp. 12, 52). The authors acknowledge that they "do not establish whether the same behavior holds at substantially larger model scales" (p. 9), and I could not find an explanation of why this small setting is a good proxy for the question. The contrastive setup itself is also weak: the negatives for each anchor are sampled once and then held fixed during training (p. 52), and the history is simply flattened into the MLP. To me the setting is far from the problem the paper wants to verify. The idea is also close to existing work, since restricting positives by temporal proximity is what neighborhood-based methods already do (Tonekaboni et al., 2021; Yèche et al., 2021; p. 2), and the proposed fix works equally well with random groupings (footnote 1, p. 2).

  2. I feel the results themselves show that the learned representations do not work, which makes the claimed loss of information hard to interpret. In the authors' own words, "No learned representation in this paper exceeds the raw history block fed to the same decoder" (p. 20). PCA with 32 dimensions exceeds the strongest learned representation by +0.21 and +0.18 (p. 47), and the authors note that the untrained ReLU MLP used as the main floor itself discards information, so that "the appropriate learning-free reference is PCA" (p. 47). The 4x encoder is at best indistinguishable from random initialization (p. 53), and on GLOBEM the contrastive encoder removes only 0.14% of baseline error against 20.31% for PCA-32 (p. 39). The paper reports these points openly, but they mean that the main finding is a loss of information from representations that never carried much of it. Without a representation that works, I do not see a major improvement or a conclusion strong enough for a top-tier ML conference.

Other weaknesses:

  • The record-span effect is clear only with the 4x encoder. With the base encoder it drops to -11% and -7%, with an interval reaching zero in the single-resident cohort (pp. 25-26), and Section 4.1 does not say which encoder it uses. It is also not tested on GLOBEM (p. 38).
  • The reproducibility statement says all effects are averaged over five seeds (p. 9), but Appendices E.7 and I.2 use three (pp. 34, 50).
  • The robustness checks in Appendices D.6 and H.2 use the build described as "the headline figure" (0.0971 to 0.0198, p. 27), not the nested construction reported in Section 4.1 (0.1410 to 0.0944).
  • Several cross-references point to content that is not there (for example, Appendix H.6 is cited on p. 7 for "move far from initialization"), "regime gate" is never defined (p. 39), and code function names appear in the text (pp. 20, 31).
  • Many comparisons are reported without any correction for multiple testing.
Questions for authors
  1. Is there any setting in which the learned representations significantly exceed PCA or the raw history block? If not, what does the loss of state information mean in practice?
  2. Why is a 40k-parameter MLP with fixed negatives a good proxy for the large longitudinal and wearable foundation models that motivate the paper? Results with a standard contrastive setup (in-batch negatives, a temporal encoder) or with a larger pretrained model would help.
  3. (Clarification) Which encoder does Section 4.1 use, and does the record-span effect hold with the base encoder?
Reviewer_cMtAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies an important problem in longitudinal contrastive learning. When we use longer records, treating all observations from the same person as positive pairs may cause the model to lose information about how that person changes over time.

The authors study the difference between record length and the time distance between positive pairs. They show that longer records can reduce useful information about a person's changing state, even when positive pairs are kept close in time. They also show that using other observations from the same person as negative pairs can help recover this information.

The experiments use longitudinal behavioral data from DETECT, with an additional evaluation on GLOBEM.

Critical strengths
  1. I think the paper studies an important problem that can easily be overlooked in longitudinal learning. The difference between having more historical data and making the model invariant to more of that history is interesting. I especially like the finding that even observations that were never used as positive pairs can become more similar as records get longer.

  2. The experiments are carefully designed. The authors test record length and positive-pair distance separately, and include several useful controls. I also appreciate the additional experiment on GLOBEM with 199 participants, which supports the main finding on a different dataset.

Critical weaknesses
  1. My main concern is the limited generalizability of the results. Most experiments are based on the DETECT dataset, which has a relatively small number of independent participants and households. The GLOBEM experiment supports the effect of positive-pair distance, but it does not test the longer-record effect or the proposed within-person negative sampling approach. It is therefore unclear whether all the findings would hold on other longitudinal datasets.

  2. I also have some concerns about the evaluation setup. Appendix E.6 explains that some positive pairs in the original experiments could overlap with the prediction target or include observations from the test period. The authors repeat these experiments with cleaner data selection and report similar results, which is reassuring. However, I think the authors should present these corrected experiments as the main results rather than only as additional analysis.

Questions for authors
  1. Can the authors clarify why the original evaluation setup was kept as the main experiment even though the corrected experiments are available? Would it be possible to use the results with stricter data separation throughout the main paper?

  2. Have the authors tested the longer-record effect or within-person negative sampling on another independent dataset? The GLOBEM results are useful, but they only confirm one part of the findings. I think additional evidence would help support the broader conclusions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.