- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
Problem
What is the problem/question?
How should we effectively scale action-conditioned trajectory generation models over wearable signals data, considering axes such as population size, training context (max sequence length seen during training), model capacity, and inference context (max sequence length seen during inference)?
Why is the problem/question important/impactful?
Understanding scaling laws makes it much easier to build the best possible model at a given compute and data budget, and highlight the most important directions the field should study / resources the field should curate to maximally improve model performance. This field is of increasing interest in the context of large language models, but has been less well studied in other settings.
Why is the problem/question hard?
Determining scaling laws that effectively extrapolate to unseen dataset sizes and model capacities requires nuanced understanding of the appropriate metrics to use for population size (e.g., number of unique sequences vs. number of unique patients) and model capacity (width, depth, attention heads) and are often expensive to run, as you need to fit many runs to determine their scaling relationships. You also need to use the right form of the scaling law, which can be challenging as it may change depending on the dataset and scaling regime in question.
What naive solutions have or should have been tried to solve this problem/question?
Simply run everything and fit a wide battery of laws and identify those from used in the literature to date that fit best.
Why are these naive solutions insufficient?
They aren't, usually, they just may not have been done before. Though, that is undercut by some of the novelty concerns.
Solution
How do the authors propose to solve it (in brief)
The authors perform a grid over # of patients, training context seen per patient, inference context allowed, and model capacity (on an unclear scale).
What experimental questions do the authors answer to show that they solve it?
- Whether their trained model shows accurate forecasting capabilities over long time horizons (up to 6 mo).
- Whether learned representations show utility in linear probing in addition to zero-shot forecasting capabilities.
- Whether increasing number of subjects vs. amount of data per subject matters more
- Whether higher-capacity models gain more or less from more context
- Whether scaling on various axes matter more for general forecasting capabilities or tasks of greater real-world impact?
- Whether contextual benefit matters more for inference capabilities or in training quality
What key experimental questions are missing?
- isoflop style analyses which ask how compute should be divided between observing more training sequences, longer sequences, or over larger model parameters at a given compute budget.
- What (if any) explicit scaling laws can be fit to the data that permit extrapolation of loss to larger settings.
- Whether training loss reliably correlates with task-specific performance.
What results are revealed from those experiments, and do they support the author's claims?
- The authors report results of their model's forecasting capabilities against a trailing 30d baseline. It is unclear how other kinds of models would perform on this, however, rendering it hard to make judgments of individual model performance.
- The authors report that forecasting is better on users with longer history when longer history is leveraged for inference, which may or may not mean that amount of data per subject is more valuable than number of subjects (see below).
- The authors report that larger capacity models perform better, especially when more context is leveraged.
- That scaling gains are larger in targeted tasks, rather than general forecasting.
Novelty
Has this been done before? If not, why not?
The authors claim that they produce the first "population-scale wearable world model for forecasting longitudinal physiological trajectories conditional on future behavior". I do not think this is true -- or that if it is true it likley has too many conditionals to be as meaningful.
There certainly are numerous generative, AR models over very large datasets of wearable data, including SensorFM and this paper (https://arxiv.org/pdf/2307.16664) that used a dataset of >2B user-days of data.
If the primary aspect of novelty here is simply that these models are not "conditioned on future behavior", I'm not sure that is as meaningful given (by the author's own statement) this model still is not a causal / RL model, and merely predicts associative effects that leverage this conditional element.
The authors also state that "Longitudinal... data introduce a further distinction: more data can mean more individuals or more history from each individual". This is actually not a distinction from natural language processing studies. More data can mean either more text sequences or longer text sequences, so I don't feel that this introduces a major novelty differential either. This question has actually been directly analyzed in the longitudinal data space as well, in "Scaling Wearable Foundation Models", which has a section entitled "Is scaling subjects or... hours per subject more helpful?"
Are there papers / references that are natural for this that the authors should highlight, and do they?
The authors do seem to cite most of the right works, but underappreciate/emphasize their releavnce, I think.
Limitations & Societal Context
Limitations
What limitations are there in this approach, and do the authors appropriately identify them?
The authors identify limitations sufficiently.
Ethical Concerns
Are there any ethical concerns with this approach, and do the authors appropriately identify them?
None.
Critical strengths
Key Strengths
These are reasons I would advocate the paper be accepted
- I think that scaling law research is highly impactful and helps make methods research more accessible and translatable to deployment.
- I think that separating out context-size, model capacity, and training dataset size is useful for furthering understanding of scaling laws in this setting.
- I like the specialization of generic and task-specific forecasting analysis.
Minor Strengths
These are strengths, but not sufficient alone to warrant advocating acceptance
- I think the paper is generally relatively clear, and I appreciate the organization of the results section around the key questions the authors evaluate.
Critical weaknesses
Key Weaknesses
These are reasons I would advocate the paper be rejected
- This paper tries to fit two framings at once: (1) Presenting a new "world model" over wearables data at a very large scale, capable of long-horizon, action-conditioned prediction and (2) a detailed analysis of scaling behaviors of said model / wearables models generally. This is a problem because both of these goals have different desiderata, and doing them simultaneously risks (and here may achieve) doing neither well. For (1), a key comparison would be against other existing wearables foundation models in downstream task quality, and (2) warrants studying over an established model family / architecture such that inductive insight offered by scaling law analyses can be useful for readers, and potentially a different focus or greater experimental depth than is presented here. Better specializing to one or the other (or trying to expand on and do both properly) would require both framing adjustments and experimental adjustments throughout.
- In the context of this as a scaling law study, it has greatly limited novelty relative to "Scaling Wearable Foundation Models". It's major differentiator against that study is (a) the question of how the amount of inference context matters relative to training contexts (which I have concerns about, mentioned below), (b) the analysis of how gains differ between general forecasting and cohort-specific forecasting, and (c) whether previous findings generalize to longer horizon models/tasks. While these are meaningful, they don't necessarily meet the novelty bar in my opinion for this venue.
- I do not think the inference context study is as interesting/impactful as advertised, and I believe it has some validity issues. * In terms of interestingness/impactfulness, I think generally speaking you would expect that in a large data/capacity regime, increasing sequence lengths will almost always lead to improved performance, and the choice about how much to increase sequence lengths is more a question of computational resource constraints. Similarly, if you train a model at a max sequence length of M, there is no reason beyond computational efficiency not to use an inference context length of M-1 (e.g., the full possible save the slot needed for the generated next token), so the question of how much improvement does increasing M make if you still restrict inference context to fixed k << M is somewhat strange to me. Had you done something like assessed at given total compute budgets (e.g., ISOflop curves) what max sequence length M is right, as increasing M will both increase performance and compute cost for a given dataset size and model capacity, that would be more interesting, but I don't see such a result here. * In terms of validity, critically, restricting samples to only those with an observed degree of inference context is not a random subsample of the population -- forecasting on users who have more or less total data observed may be easier or harder. Instead, you should take a population that all have the same degree of longitudinal data observed, then subsample inference context length, such that no users are being included/excluded differently at different context lengths, and the only changing variable is how much context they can use at inference time.
- You don't actually deliver any scaling laws (or demonstrate that they do not successfully fit in your setting) for your model type, limiting how much users can take away from this study. I'd want to see you do something like fit Chinchilla or other style of scaling law to your data, such that you can comment on extrapolation performance, how fits differ systematically across measures or settings, etc. Even if the fits don't work well, identifying that there are gaps in the scaling law literature for this style of model would be useful; without them, your results are limited in how much users can derive actionable insight from them for future studies.
Minor Weaknesses
These are weaknesses, but not sufficient alone to warrant advocating rejection
- I wouldn't frame your cohort specific results as showing the limitations of population level aggregation -- it is just an evaluation on different tasks. One task is forecasting from an arbitrary event, the other is a more specific prediction task. This doesn't limit the value of these observations, to be clear, I'm just suggesting a slightly different way to present them.
Questions for authors
Action Items for Rebuttal
Things that, if addressed, may change my score
If you could clarify that there is not the validity issue regarding inference context length and better identify why that arm provides material inductive value vs., e.g., presenting a unified isoflop curve view, and could further fit actual scaling laws or present real isoflop curves, that might make me improve my score, but it is unlikely to reach an accept, given the novelty concerns would remain.
Questions
Things that I want to know, but are unlikely to result in me changing my score
2 comments
Response
I think you're being a bit too dismissive and narrow when thinking about isoFLOP analysis. IsoFLOP analysis isn't just about "how to spend compute on a given dataset" -- instead, it says "if you have a given training compute budget, how should you trade off the various choices one needs to make in model design that change how much compute a single gradient pass costs vs. how many passes you get to make. It is very natural, in fact, to incorporate context size into isoFLOP analysis. If I can spend 10^20 FLOPs training my model, do I want to spend that with more passes over shorter sequences? Or fewer passes over longer sequences? Or even fewer passes over longer sequences with a bigger model? IsoFLOP analysis identifies how to anticipate what choices from that set one should make to yield minimal loss (or any metric that can be evaluated equivalently across settings) over larger compute budgets than have been tested, such that future models can be built more efficiently. I'd strongly encourage you to take more of that perspective to heart in future revisions of the work.
Unfortunately, as I indicated in my original review, I think even with the rebuttal this is insufficient to change my score, but best of luck with your submission going forward!
Response to Reviewer YtwY
We thank the reviewer for a thoughtful review and for engaging with the paper. The global response covers the new baselines and how we use action conditioning. Below we address novelty relative to LSM and SensorFM and the question of isoFLOP analysis, both of which we believe stem from a framing we did not make clear in the paper.
The question we study. We ask which data to prioritize when building longitudinal health models, longer histories or additional users. In health, data is usually the binding constraint. The number of people and the length of their histories are set by who has worn a device and for how long, and more compute cannot add to either. IsoFLOP analysis answers a related and valuable question about how to spend compute on a given dataset. Our results show that breadth saturates near one million users while depth continues to help through 365 days. For example, the M model trained on 1.7M users with a full year of history performs the same as the M model trained on all 5.4M users (median R² of 0.655 for both across 308 panels), and at matched user-days the deeper model wins on 96% and 93% of panels at the two budgets. Moreover, these trends differ based on the evaluation cohort we use, where we show that scaling gains concentrate around physiological events. Hence, no single functional form can describe the scaling behavior, and we show that it should be assessed across targets, horizons, and periods of physiological change.
Novelty relative to LSM and SensorFM. These papers established scaling behavior for wearable foundation models, and we will discuss them more prominently. Neither varies how much personal history the model conditions on. LSM (Narayanswamy et al., 2025) trains on 5-hour windows from 165K people. Its subjects-versus-hours experiment holds the window at five hours, so more hours per subject adds training windows without lengthening the context, and the authors hypothesize that subject diversity may matter more as windows grow to days. SensorFM (Narayanswamy et al., 2026) samples 10 weeks per participant from 5M people, uses a 24-hour context window, and scales model and data jointly. We vary conditioning history from 7 to 365 days, separating training context from inference context, and cross both with breadth (10K to 5.4M users) and capacity (1.3M to 303M parameters), for forecasts up to 180 days. To our knowledge, this is the first test of depth versus breadth in this regime, which LSM left open, and our findings differ from the recommendation to scale both together.
Action conditioning. We agree that the model is not yet causally validated, and the paper states this. Conditioning on behavior still allows the model to predict how a person's trajectory responds to a different plan, which a static encoder cannot do. The global response gives population-level effects of alcohol and activity, along with an estimate for each user.
We will revise the introduction and related work to make these distinctions explicit and add the new results. We thank the reviewer for these comments, which have made the paper stronger.