acceptodds
Under review as a conference paper at ICLR 2027

CardiacState: Learning Cyclic Cardiac State through Predictive Video Pretraining

Abstract

Predicting how an observed physical system will evolve requires inferring the underlying state that governs its motion. While controllable video generation relies on external action or text signals, action-free predictive models leave this conditioning implicit within high-dimensional representations. Autonomous biological systems, such as the beating heart, lack external controllers—their driving dynamics unfold continuously within the visual stream itself. Yet, core clinical tasks like keyframe localization and Left Ventricular Ejection Fraction (LVEF) estimation depend heavily on tracking this physiological state. We introduce CardiacState, a joint-embedding predictive architecture (JEPA) that addresses this by formulating physiological progression as an explicit, self-derived conditioning state on a continuous cyclic manifold . By imposing a causal temporal bottleneck that denies the future-feature predictor direct access to historical frames, CardiacState forces all temporal context to route exclusively through this state alongside physical elapsed time . Pretrained on 1.88 million clinical echocardiogram videos without external text, action labels, or hardware ECG gating, CardiacState advances both feature representation and structural interpretability. Its frozen visual representations achieve LVEF MAE on EchoNet-Dynamic and on EchoNet-Pediatric—outperforming prior self-supervised baselines ( and ). Furthermore, unlike implicit representations, this explicit cyclic state directly enables non-parametric keyframe localization across diverse acoustic views via phase-anchor matching, reducing mean localization error from frames to frames. Controlled ablations show that cardiac progression regularization is important for learning a useful cyclic coordinate, while the temporal bottleneck provides further gains in localization and representation quality. These results support cyclic state conditioning for predictive learning in quasi-periodic systems.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Echocardiography, Medical video analysis, Cardiac functional assessment, Self-supervised video representation learning, Temporal representation learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UbwWOpenReview ↗
Rating
4(+1)
Confidence
4(+1)
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper introduces CardiacState, a joint-embedding predictive architecture for echocardiograms. It infers an explicit cyclic state on S1S^1 from visual history, and combines this state with current-frame features and physical elapsed time to predict future representations. A causal temporal bottleneck restricts the future predictor from directly accessing historical frames, forcing temporal context through the low-dimensional cyclic state.

Core Claim: In action-free dynamic systems, combining a structured low-dimensional state with an information bottleneck enables predictive models to learn physically grounded endogenous dynamic states directly from visual streams.

Critical strengths
  1. Well-motivated endogenous state formulation: Unlike action-conditioned world models that rely on external actions, cardiac motion lacks external control signals. CardiacState infers an internal state and constrains it to an S1S^1 manifold. Importantly, this cyclic state is not just a passive auxiliary loss; it actively conditions future feature prediction.

  2. Information bottleneck as a general representation principle: By denying the predictor direct access to historical visual tokens (K=1K=1), historical dynamics are forced through the explicit state channel. In Table 3, progression regularization drops localization MAE from 18.396 to 2.107 frames, and enforcing K=1K=1 further reduces it to 2.002 frames while improving frozen LVEF MAE from 5.028% to 4.867%. These results support combining structured states with temporal bottlenecks.

Critical weaknesses
  1. Cyclic states are not novel, and the impact of the bottleneck is relatively marginal: Using S1S^1 cyclic representations for cardiac cycles has been explored in prior work (e.g., DeepHeartBeat, OrbitalEcho). The primary novelty here is using S1S^1 as a condition under an architectural bottleneck. However, the ablations in Table 3 show that the dominant gain comes from the progression loss Lcardiac\mathcal{L}_{cardiac} (MAE 18.396 →\to 2.107), whereas reducing context from K=4K=4 to K=1K=1 yields a much smaller increment (2.107 →\to 2.002). This indicates the main improvement stems from explicit temporal regularization rather than the novel bottleneck design itself.

  2. The bottleneck does not guarantee state utilization: Restricting direct frame access forces history through the state pathway if historical information is used, but it does not prove the predictor relies on this channel. The model could still rely primarily on the current frame feature and elapsed time Δτt\Delta\tau_t, while the cyclic state organizes independently under Lcardiac\mathcal{L}_{cardiac}. The experiments do not fully rule out this alternative explanation. Additionally, keyframe localization evaluates error against the nearest predicted event; increasing the number of predicted events can naturally lower this distance without necessarily reflecting better physiological phase alignment.

Questions for authors
  1. Under fixed K=1K=1, what happens to future feature prediction and downstream LVEF if the cyclic state ctc_t is ablated, shuffled across time, or replaced with a fixed periodic signal? Does the predictor strictly rely on the state inferred from visual history?

  2. Does the learned cyclic state reliably correspond to true cardiac cycles? After controlling for the number of predicted events and adopting a more stringent event-matching protocol, does the advantage in keyframe localization still hold?

7 comments

We thank Reviewer 3 for the sharp, focused questions regarding the contribution of the K=1K=1 bottleneck, causal state utilization by the predictor, and event-count control in keyframe evaluation.

1. Value of the K=1K=1 Temporal Bottleneck Beyond Lcardiac\mathcal{L}_{\text{cardiac}} (W1)

We thank the reviewer for raising this important distinction. To isolate the contributions of explicit cyclic-state conditioning and the causal bottleneck from standard feature prediction, we trained a matched time-conditioned K=4K=4 JEPA baseline without the cardiac-state module or Lcardiac\mathcal{L}_{\text{cardiac}} under identical initialization, pretraining data, sampling, and compute budget. Because this baseline produces no phase representation, keyframe localization is not applicable (N/A), and we evaluate its frozen visual representations on LVEF estimation alongside our ablation variants:

MethodEchoNet-Dynamic MAE ↓\downarrowEchoNet-Dynamic MSE ↓\downarrowEchoNet-Pediatric MAE ↓\downarrowEchoNet-Pediatric MSE ↓\downarrowLocalization Mean MAE (frames) ↓\downarrow
K=4K=4, JEPA w/o cardiac state5.304±0.2565.304 \pm 0.25652.131±2.32852.131 \pm 2.3285.041±0.2785.041 \pm 0.27857.614±7.03257.614 \pm 7.032N/A
K=4K=4, w/o Lcardiac\mathcal{L}_{\text{cardiac}}5.204±0.1335.204 \pm 0.13350.154±2.98050.154 \pm 2.9805.070±0.1915.070 \pm 0.19158.648±6.11058.648 \pm 6.11018.396±0.57018.396 \pm 0.570
K=4K=4, w/ Lcardiac\mathcal{L}_{\text{cardiac}}5.028±0.1245.028 \pm 0.12445.549±2.38545.549 \pm 2.3854.811±0.1754.811 \pm 0.17550.835±5.39750.835 \pm 5.3972.107±0.0272.107 \pm 0.027
K=1K=1, w/ Lcardiac\mathcal{L}_{\text{cardiac}}4.867±0.1214.867 \pm 0.12142.554±2.24942.554 \pm 2.2494.725±0.1754.725 \pm 0.17549.599±5.18849.599 \pm 5.1882.002±0.0232.002 \pm 0.023
  • A bare cyclic state is insufficient without echocardiography-specific inductive bias: While S1S^1 parameterizations have appeared in prior pixel-reconstruction models, our ablation (K=4K=4, w/o Lcardiac\mathcal{L}_{\text{cardiac}}) demonstrates that simply introducing a cyclic state module into latent predictive pretraining fails completely—the state suffers from representation collapse, remaining nearly stationary around a narrow angle (Figure 2, top; localization MAE 18.39618.396 frames), which is functionally equivalent to having no usable state output (analogous to N/A). Making cyclic conditioning work in a predictive architecture requires coupling S1S^1 with our echocardiography-grounded inductive bias (Lcardiac\mathcal{L}_{\text{cardiac}}, which enforces physical-time-scaled forward progression while accommodating heart-rate-dependent subphase velocities) and the K=1K=1 bottleneck.
  • Interpreting keyframe localization against a valid baseline: Because the 18.39618.396-frame error reflects a collapsed readout rather than a functional baseline, the meaningful external reference under the protocol-matched A4C evaluation is OrbitalEcho at 2.2452.245 frames MAE (Table 2). Relative to this benchmark, K=4 w/ LcardiacK=4\text{ w/ }\mathcal{L}_{\text{cardiac}} achieves 2.1052.105 frames (−0.140-0.140), and tightening the bottleneck from K=4K=4 to K=1K=1 further reduces A4C error to 1.9851.985 frames (−0.120-0.120, Tables 2 and 5)—accounting for 46.2% of our total localization gain over OrbitalEcho, while also eliminating local phase stalls (Figure 2, middle vs. bottom).
  • Substantial gains on frozen-backbone LVEF representation: The impact of the K=1K=1 bottleneck is equally pronounced on downstream functional assessment. On EchoNet-Dynamic, adding Lcardiac\mathcal{L}_{\text{cardiac}} reduces LVEF MAE relatively by 3.382%3.382\% (5.204%→5.028%5.204\% \to 5.028\%), while enforcing the K=1K=1 bottleneck yields an additional 3.202%3.202\% relative reduction (5.028%→4.867%5.028\% \to 4.867\%, and MSE from 45.54945.549 to 42.554%242.554\%^2)—accounting for nearly half (47.8%) of the total LVEF accuracy improvement. Denying multi-frame persistence shortcuts forces prediction gradients to encode global ventricular dynamics inside the visual backbone EϕE_\phi.

2. Direct Proof that the K=1K=1 Predictor Relies on the Inferred State ctc_t (W2 & Q1)

For downstream LVEF estimation, we evaluate the frozen visual encoder EϕE_\phi with the state module and predictor detached; the state pathway is trained jointly with EϕE_\phi during pretraining (and used explicitly for ED/ES localization), so its effect on LVEF is mediated through the enriched visual features (removing the state module during pretraining degrades Dynamic LVEF MAE from 4.867%4.867\% to 5.304%5.304\%).

To directly test whether the next-feature predictor relies on the inferred state ctc_t rather than ignoring it in favor of (Ft,Δτt)(F_t, \Delta\tau_t), we froze each pretrained model and rotated the current cardiac state ct∈S1c_t \in S^1 at inference while keeping all other inputs fixed. Below we report the feature RMSE between predictions made with the rotated state and those made with the original state across all three datasets:

EchoNet-Dynamic

RotationK=4K=4, w/o Lcardiac\mathcal{L}_{\text{cardiac}}K=4K=4, w/ Lcardiac\mathcal{L}_{\text{cardiac}}K=1K=1, w/ Lcardiac\mathcal{L}_{\text{cardiac}}
10∘10^\circ1.680±0.0871.680 \pm 0.0871.674±0.0951.674 \pm 0.0952.162±0.1212.162 \pm 0.121
20∘20^\circ1.735±0.0921.735 \pm 0.0921.740±0.0961.740 \pm 0.0962.482±0.1402.482 \pm 0.140
30∘30^\circ1.770±0.0951.770 \pm 0.0951.793±0.0981.793 \pm 0.0982.781±0.1612.781 \pm 0.161
60∘60^\circ1.844±0.0981.844 \pm 0.0981.889±0.1021.889 \pm 0.1023.822±0.2463.822 \pm 0.246
90∘90^\circ1.898±0.1021.898 \pm 0.1021.948±0.1051.948 \pm 0.1054.769±0.4134.769 \pm 0.413
120∘120^\circ1.943±0.1081.943 \pm 0.1081.962±0.1051.962 \pm 0.1055.201±0.6565.201 \pm 0.656

EchoNet-Pediatric

RotationK=4K=4, w/o Lcardiac\mathcal{L}_{\text{cardiac}}K=4K=4, w/ Lcardiac\mathcal{L}_{\text{cardiac}}K=1K=1, w/ Lcardiac\mathcal{L}_{\text{cardiac}}
10∘10^\circ1.725±0.1141.725 \pm 0.1141.862±0.1231.862 \pm 0.1232.409±0.1402.409 \pm 0.140
20∘20^\circ1.766±0.1111.766 \pm 0.1111.936±0.1251.936 \pm 0.1252.781±0.1632.781 \pm 0.163
30∘30^\circ1.800±0.1181.800 \pm 0.1181.987±0.1251.987 \pm 0.1253.123±0.1933.123 \pm 0.193
60∘60^\circ1.873±0.1211.873 \pm 0.1212.097±0.1292.097 \pm 0.1294.242±0.3934.242 \pm 0.393
90∘90^\circ1.928±0.1311.928 \pm 0.1312.159±0.1342.159 \pm 0.1345.097±0.7165.097 \pm 0.716
120∘120^\circ1.968±0.1341.968 \pm 0.1342.173±0.1332.173 \pm 0.1335.311±0.9855.311 \pm 0.985

EchoNet-LVH

RotationK=4K=4, w/o Lcardiac\mathcal{L}_{\text{cardiac}}K=4K=4, w/ Lcardiac\mathcal{L}_{\text{cardiac}}K=1K=1, w/ Lcardiac\mathcal{L}_{\text{cardiac}}
10∘10^\circ1.473±0.1041.473 \pm 0.1041.590±0.1141.590 \pm 0.1141.919±0.1571.919 \pm 0.157
20∘20^\circ1.508±0.1071.508 \pm 0.1071.652±0.1131.652 \pm 0.1132.202±0.1792.202 \pm 0.179
30∘30^\circ1.527±0.1131.527 \pm 0.1131.707±0.1151.707 \pm 0.1152.462±0.2312.462 \pm 0.231
60∘60^\circ1.595±0.1161.595 \pm 0.1161.785±0.1261.785 \pm 0.1263.231±0.5223.231 \pm 0.522
90∘90^\circ1.638±0.1201.638 \pm 0.1201.836±0.1281.836 \pm 0.1283.698±0.8893.698 \pm 0.889
120∘120^\circ1.679±0.1201.679 \pm 0.1201.854±0.1271.854 \pm 0.1273.748±1.1793.748 \pm 1.179

The predicted features diverge systematically from the original predictions as the rotation angle increases, and the changes are larger at K=1K=1 than at K=4K=4 across all tested angles and datasets. This directly confirms that the predictor actively uses the cardiac state and relies on it substantially more under the tighter bottleneck.

3. Controlling for Detection Count in Keyframe Localization (W2 & Q2)

We moved Figure 2 from the supplement into the main paper, illustrating that the state learned by the K=1K=1 model with Lcardiac\mathcal{L}_{\text{cardiac}} follows true cardiac cycles across beats in the representative video.

In the submitted evaluation, all CardiacState variants used the same 5-frame local temporal window to suppress nearby candidate detections during ED/ES peak selection (Appendix C). To assess detection counts under sparse annotations, we estimate each video's cycle count by dividing its length in frames by twice the annotated ED–ES interval. Detection density is the number of predicted ED and ES events divided by the estimated cycle count.

\text{detection density} = \frac{N_{\text{predictions}}}{\widehat{N}_{\text{cycles}}}.$$ One ED and one ES event per beat corresponds to an expected density of approximately $2.0$. Across every acoustic view, our $K=1$ model averages **$2.280 \pm 0.934$** detections per estimated cycle and achieves lower localization MAE than $K=4$ while generating **fewer** detections per estimated cycle—ruling out over-detection as the source of localization accuracy. A beat-by-beat assessment under a one-to-one event-matching protocol would require exhaustive multi-cycle ED/ES annotations, which we note in Section 5 as an important direction for future work. | View | Method | MAE $\downarrow$ | RMSE $\downarrow$ | $R^2$ $\uparrow$ | Estimated cycles | Estimated cycle frame length | Estimated detection density | | :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | A4C | $K=4$, w/o $\mathcal{L}_{\text{cardiac}}$ | $28.475 \pm 0.997$ | $49.082 \pm 1.395$ | $-0.429 \pm 0.094$ | $5.50 \pm 3.13$ | $33.91 \pm 10.98$ | $2.217 \pm 1.979$ | | A4C | $K=4$, w/ $\mathcal{L}_{\text{cardiac}}$ | $2.105 \pm 0.029$ | $2.518 \pm 0.028$ | $0.9962 \pm 0.0002$ | $5.50 \pm 3.13$ | $33.91 \pm 10.98$ | $2.200 \pm 0.708$ | | A4C | $K=1$, w/ $\mathcal{L}_{\text{cardiac}}$ | **$1.985 \pm 0.024$** | **$2.325 \pm 0.022$** | **$0.9967 \pm 0.0002$** | $5.50 \pm 3.13$ | $33.91 \pm 10.98$ | **$2.137 \pm 0.692$** | | PSAX | $K=4$, w/o $\mathcal{L}_{\text{cardiac}}$ | $15.183 \pm 0.883$ | $27.059 \pm 1.690$ | $0.501 \pm 0.137$ | $2.67 \pm 1.67$ | $39.42 \pm 21.30$ | $2.724 \pm 2.971$ | | PSAX | $K=4$, w/ $\mathcal{L}_{\text{cardiac}}$ | $2.083 \pm 0.047$ | $2.573 \pm 0.051$ | $0.9953 \pm 0.0011$ | $2.67 \pm 1.67$ | $39.42 \pm 21.30$ | $2.546 \pm 1.370$ | | PSAX | $K=1$, w/ $\mathcal{L}_{\text{cardiac}}$ | **$2.027 \pm 0.045$** | **$2.452 \pm 0.054$** | **$0.9958 \pm 0.0010$** | $2.67 \pm 1.67$ | $39.42 \pm 21.30$ | **$2.518 \pm 1.332$** | | PLAX | $K=4$, w/o $\mathcal{L}_{\text{cardiac}}$ | $11.531 \pm 1.055$ | $24.956 \pm 2.077$ | $0.572 \pm 0.101$ | $5.20 \pm 1.83$ | $34.68 \pm 8.29$ | $3.213 \pm 1.558$ | | PLAX | $K=4$, w/ $\mathcal{L}_{\text{cardiac}}$ | $2.135 \pm 0.061$ | $2.597 \pm 0.058$ | $0.9954 \pm 0.0008$ | $5.20 \pm 1.83$ | $34.68 \pm 8.29$ | $2.327 \pm 0.562$ | | PLAX | $K=1$, w/ $\mathcal{L}_{\text{cardiac}}$ | **$1.994 \pm 0.049$** | **$2.358 \pm 0.046$** | **$0.9962 \pm 0.0006$** | $5.20 \pm 1.83$ | $34.68 \pm 8.29$ | **$2.185 \pm 0.528$** | | Avg | $K=4$, w/o $\mathcal{L}_{\text{cardiac}}$ | $18.396 \pm 0.570$ | $33.699 \pm 1.011$ | $0.215 \pm 0.065$ | $4.46 \pm 2.63$ | $36.01 \pm 14.84$ | $2.718 \pm 2.285$ | | Avg | $K=4$, w/ $\mathcal{L}_{\text{cardiac}}$ | $2.107 \pm 0.027$ | $2.563 \pm 0.028$ | $0.9956 \pm 0.0005$ | $4.46 \pm 2.63$ | $36.01 \pm 14.84$ | $2.357 \pm 0.958$ | | Avg | $K=1$, w/ $\mathcal{L}_{\text{cardiac}}$ | **$2.002 \pm 0.023$** | **$2.378 \pm 0.024$** | **$0.9962 \pm 0.0004$** | $4.46 \pm 2.63$ | $36.01 \pm 14.84$ | **$2.280 \pm 0.934$** |
Reviewer_ZjcFOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CardiacState, a joint-embedding predictive architecture (JEPA) designed to learn explicit cyclic physiological states from echocardiography videos without external text supervision, action labels, or ECG gating. The method introduces a self-derived cyclic conditioning state and a causal temporal bottleneck that forces future-feature prediction to rely on the learned state and elapsed time rather than directly accessing historical frames. Pretrained on 1.88 million clinical echocardiography videos, CardiacState evaluates the learned representations on left ventricular ejection fraction (LVEF) estimation and cardiac keyframe localization. The authors report improvements over prior self-supervised learning baselines on public adult and pediatric echocardiography datasets, and demonstrate that the learned cyclic state supports phase-anchor-based keyframe localization. Ablation studies further investigate the contributions of cardiac progression regularization and the temporal bottleneck.

Critical strengths
  1. The paper addresses a clinically relevant problem by learning cardiac temporal dynamics directly from large-scale echocardiography videos without requiring explicit ECG signals or manually annotated cardiac phases. This could reduce dependence on additional clinical annotations and specialized acquisition protocols.
  2. The proposed explicit cyclic state provides an interpretable representation of cardiac progression and supports both downstream representation learning and keyframe localization within a unified framework. The use of a large pretraining dataset and evaluation on adult and pediatric echocardiography benchmarks is also a strength.
Critical weaknesses
  1. The methodological novelty appears moderate. Predictive representation learning, JEPA architectures, latent-state conditioning, cyclic representations, and temporal information bottlenecks have been explored in related areas. The paper should more clearly distinguish its fundamental contributions from existing approaches and explain what new technical insight is provided by combining these concepts for cardiac video analysis.

  2. The evidence supporting the learned cyclic state requires further clarification. Although the proposed state is intended to represent physiological progression, it remains unclear how reliably it corresponds to actual cardiac phases across different patients, acoustic views, and acquisition conditions. More direct validation against annotated cardiac phases or independent physiological measurements would strengthen the claims regarding interpretability and clinical relevance.

  3. The experimental presentation and methodological analysis could be strengthened. The abstract reports improvements in LVEF estimation and keyframe localization, but the magnitude of these improvements and their statistical significance are not clear from the description. More comprehensive comparisons with relevant temporal modeling and self-supervised learning methods, together with analyses of robustness and computational cost, would help establish the practical advantages of CardiacState.

  4. The manuscript requires substantial improvement in presentation. The abstract includes technical terminology and mathematical notation that may make the central idea difficult to follow, while the framework illustrations, experimental figures and tables, and detailed textual explanations need greater clarity. A more intuitive explanation of the causal temporal bottleneck and the role of the cyclic state would make the technical contributions and experimental evidence easier to assess.

Questions for authors
  1. What is the fundamental methodological novelty of CardiacState compared with existing JEPA-based predictive learning, latent-state conditioning, and cyclic representation learning methods? Can the authors identify the specific technical contribution that cannot be achieved by straightforwardly combining these existing approaches?

  2. How do the authors verify that the learned cyclic state corresponds to meaningful physiological progression rather than merely capturing temporal regularities in echocardiography videos? Have they evaluated its consistency with annotated cardiac phases or its robustness across patients, acoustic views, and acquisition conditions?

  3. How does the causal temporal bottleneck affect representation quality and predictive performance compared with alternative architectures that allow direct access to historical frame features? Can the authors provide quantitative ablations that isolate the contribution of the bottleneck from that of cyclic-state regularization?

  4. The paper reports improvements in LVEF estimation on both adult and pediatric datasets. How consistent are these improvements across different patient subgroups and imaging conditions, and are they statistically significant compared with the strongest self-supervised baselines?

  5. Could the authors improve the framework diagram and experimental figures to make the information flow, cyclic-state learning process, and keyframe localization procedure easier to understand? In addition, could the abstract and main text be revised to reduce unnecessary technical notation and provide clearer explanations of the proposed mechanisms?

7 comments

We thank Reviewer 2 for the constructive feedback on articulating our methodological novelty, validating physiological interpretability, reporting statistical significance and computational costs, and improving manuscript clarity.

1. Fundamental Methodological Novelty & Technical Insight (W1 & Q1)

We agree that the individual building blocks—joint-embedding prediction, circular manifolds, and temporal bottlenecks—have been studied in separate contexts. Our contribution is not proposing a new standalone architectural operator, but rather formulating a new conditioning paradigm for autonomous biological systems and uncovering the specific technical insight required to make it work in cardiac video analysis:

1.1. Conceptual Formulation — Self-Derived Conditioning for Autonomous Periodic Systems: Existing predictive world models fall into two regimes: action-conditioned models (e.g., robotics/gaming), which rely on externally supplied control signals ata_t, and action-free video JEPAs (e.g., V-JEPA, EchoJEPA), which leave temporal dynamics implicit inside opaque, high-dimensional visual embeddings. Autonomous biological processes like the beating heart have no external controller, yet are driven by an explicit, low-dimensional physiological state. CardiacState bridges these regimes by formulating cardiac progression as an endogenous conditioning variable inferred strictly from visual history to drive future-feature prediction.

1.2. Technical Insight — Resolving the Phase-Ambiguity vs. Persistence-Shortcut Dilemma: Straightforwardly combining JEPA, an S1S^1 state, and a bottleneck for echocardiography fails because two opposing pathologies arise in cardiac ultrasound:

  • Multi-frame persistence shortcuts (K>1K>1): Because adjacent echocardiographic frames share nearly identical myocardial geometry, a predictor with direct access to visual history can extrapolate future features via local continuity without coupling to the cyclic state—leading to local phase stalls (Figure 2, middle) and suboptimal LVEF features.
  • Single-frame phase ambiguity (K=1K=1) & state collapse: Restricting the predictor to the current frame FtF_t eliminates local shortcuts, but isolated frames are visually ambiguous (mid-systole and mid-diastole share nearly identical ventricular shapes). Yet simply attaching an S1S^1 bottleneck to a JEPA predictor fails completely due to representation collapse (K=4K=4 w/o Lcardiac\mathcal{L}_{\text{cardiac}} collapses to a near-stationary angle, yielding 18.39618.396 frames localization MAE, Figure 2 top).
  • Why the coupling is essential: Our key insight is that removing direct visual history (K=1K=1) becomes solvable—and beneficial—only when coupled with a physical-time-scaled progression objective (Lcardiac\mathcal{L}_{\text{cardiac}}). Together, they force instantaneous anatomy (FtF_t) and directional cycle phase (ct∈S1c_t \in S^1) to decouple.

1.3. Empirical & Scientific Payoff (Counterfactual Ablations): This coupling produces capabilities that neither standard JEPAs nor prior echocardiographic SSL models (e.g., EchoJEPA, Video-LAMAE, EchoPrime) achieve—learning an interpretable physiological coordinate without ECG or phase labels while simultaneously improving frozen-backbone LVEF representations:

  • Removing the state pathway (K=4K=4 matched JEPA baseline): Produces no phase coordinate (N/A localization) and worsens EchoNet-Dynamic LVEF MAE from 4.867%4.867\% to 5.304%5.304\%.

  • Removing Lcardiac\mathcal{L}_{\text{cardiac}} (K=4K=4, w/o Lcardiac\mathcal{L}_{\text{cardiac}}): State collapses (localization MAE 18.39618.396 frames; Dynamic LVEF MAE 5.204%5.204\%).

  • Relaxing the K=1K=1 bottleneck (K=4K=4, w/ Lcardiac\mathcal{L}_{\text{cardiac}}): Weakens causal reliance on ctc_t (confirmed by our state-rotation intervention in Q2), degrading both Dynamic LVEF MAE (4.867%→5.028%4.867\% \to 5.028\%) and localization MAE (2.002→2.1072.002 \to 2.107 frames).

2. Validation of Physiological Progression Across Patients, Views, and Conditions (W2 & Q2)

Figure 2 qualitatively illustrates the learned state trajectory on an external video, showing consistent alignment with ground-truth ED and ES frames across heartbeats. To evaluate generalization across patients, views, and acquisition conditions, we test the frozen model on 29,673 held-out videos (23,988 unique patients) across three external datasets with zero overlap with our pretraining corpus:

DatasetTaskUnique patientsVideosPopulationView used in our evaluation
EchoNet-DynamicLVEF estimation & ED/ES localization10,03010,030AdultsA4C for both tasks
EchoNet-PediatricLVEF estimation & ED/ES localization1,9587,643PediatricA4C & PSAX for LVEF; PSAX for localization
EchoNet-LVHED/ES localization12,00012,000AdultsPLAX

Because keyframe localization uses frozen states and dataset-level fixed phase anchors (aED,aESa_{\mathrm{ED}}, a_{\mathrm{ES}}) without a trained detection head, a generic temporal clock (e.g., frame index) could not consistently align a fixed S1S^1 direction with anatomical ED/ES events across 23,988 patients with diverse heart rates and arbitrary starting phases. Instead, CardiacState achieves nearly uniform localization accuracy across all three views (1.9851.985, 2.0272.027, and 1.9941.994 frames MAE; inter-view spread of 0.0420.042 frames, Table 2) and across the distinct FPS regimes shown in Figure 3 (∼20 FPS\sim 20\text{ FPS} in EchoNet-LVH, ∼50 FPS\sim 50\text{ FPS} in EchoNet-Dynamic, and 5050 to >100 FPS>100\text{ FPS} in EchoNet-Pediatric), alongside consistent frozen-backbone LVEF gains (Table 1).

We also tested the predictor's sensitivity to the cardiac state across all three external datasets by rotating ctc_t at inference while keeping all other inputs fixed (reporting feature RMSE between rotated and intact state predictions):

EchoNet-Dynamic

RotationK=4K=4, w/o Lcardiac\mathcal{L}_{\text{cardiac}}K=4K=4, w/ Lcardiac\mathcal{L}_{\text{cardiac}}K=1K=1, w/ Lcardiac\mathcal{L}_{\text{cardiac}}
10∘10^\circ1.680±0.0871.680 \pm 0.0871.674±0.0951.674 \pm 0.0952.162±0.1212.162 \pm 0.121
20∘20^\circ1.735±0.0921.735 \pm 0.0921.740±0.0961.740 \pm 0.0962.482±0.1402.482 \pm 0.140
30∘30^\circ1.770±0.0951.770 \pm 0.0951.793±0.0981.793 \pm 0.0982.781±0.1612.781 \pm 0.161
60∘60^\circ1.844±0.0981.844 \pm 0.0981.889±0.1021.889 \pm 0.1023.822±0.2463.822 \pm 0.246
90∘90^\circ1.898±0.1021.898 \pm 0.1021.948±0.1051.948 \pm 0.1054.769±0.4134.769 \pm 0.413
120∘120^\circ1.943±0.1081.943 \pm 0.1081.962±0.1051.962 \pm 0.1055.201±0.6565.201 \pm 0.656

EchoNet-Pediatric

RotationK=4K=4, w/o Lcardiac\mathcal{L}_{\text{cardiac}}K=4K=4, w/ Lcardiac\mathcal{L}_{\text{cardiac}}K=1K=1, w/ Lcardiac\mathcal{L}_{\text{cardiac}}
10∘10^\circ1.725±0.1141.725 \pm 0.1141.862±0.1231.862 \pm 0.1232.409±0.1402.409 \pm 0.140
20∘20^\circ1.766±0.1111.766 \pm 0.1111.936±0.1251.936 \pm 0.1252.781±0.1632.781 \pm 0.163
30∘30^\circ1.800±0.1181.800 \pm 0.1181.987±0.1251.987 \pm 0.1253.123±0.1933.123 \pm 0.193
60∘60^\circ1.873±0.1211.873 \pm 0.1212.097±0.1292.097 \pm 0.1294.242±0.3934.242 \pm 0.393
90∘90^\circ1.928±0.1311.928 \pm 0.1312.159±0.1342.159 \pm 0.1345.097±0.7165.097 \pm 0.716
120∘120^\circ1.968±0.1341.968 \pm 0.1342.173±0.1332.173 \pm 0.1335.311±0.9855.311 \pm 0.985

EchoNet-LVH

RotationK=4K=4, w/o Lcardiac\mathcal{L}_{\text{cardiac}}K=4K=4, w/ Lcardiac\mathcal{L}_{\text{cardiac}}K=1K=1, w/ Lcardiac\mathcal{L}_{\text{cardiac}}
10∘10^\circ1.473±0.1041.473 \pm 0.1041.590±0.1141.590 \pm 0.1141.919±0.1571.919 \pm 0.157
20∘20^\circ1.508±0.1071.508 \pm 0.1071.652±0.1131.652 \pm 0.1132.202±0.1792.202 \pm 0.179
30∘30^\circ1.527±0.1131.527 \pm 0.1131.707±0.1151.707 \pm 0.1152.462±0.2312.462 \pm 0.231
60∘60^\circ1.595±0.1161.595 \pm 0.1161.785±0.1261.785 \pm 0.1263.231±0.5223.231 \pm 0.522
90∘90^\circ1.638±0.1201.638 \pm 0.1201.836±0.1281.836 \pm 0.1283.698±0.8893.698 \pm 0.889
120∘120^\circ1.679±0.1201.679 \pm 0.1201.854±0.1271.854 \pm 0.1273.748±1.1793.748 \pm 1.179

On all three datasets, larger rotations produce larger changes in predicted features, with consistently greater sensitivity at K=1K=1 than at K=4K=4—confirming that the tighter bottleneck increases the predictor's reliance on ctc_t across diverse patient populations and views.

3. Statistical Significance, Subgroup Consistency, and Computational Cost (W3 & Q4)

For some published baselines, the checkpoints underlying the reported LVEF results were not publicly available, and we had access only to aggregate scores. A paired significance test requires predictions on the same test videos. We therefore tested CardiacState against DINOv3, PanEcho, and EchoPrime on both EchoNet-Dynamic and EchoNet-Pediatric using the per-video predictions from our frozen-encoder evaluation.

We used 10,000 paired bootstrap resamples per test set, drawing videos with replacement and using the same sampled indices for all models in each replicate. For MAE and MSE separately, we computed one-sided p-values from the upper tail of the null-centered bootstrap distribution of the baseline-minus-CardiacState error difference, with a plus-one correction. The alternative hypothesis was that CardiacState had lower error.

DatasetModelMAE ↓MAE p-valueMSE ↓MSE p-value
EchoNet-DynamicCardiacState4.867ref42.554ref
EchoNet-DynamicDINOv35.5069.999e-05***55.7769.999e-05***
EchoNet-DynamicPanEcho5.1910.00409959**48.5200.0151985*
EchoNet-DynamicEchoPrime5.2920.00059994***48.4710.0119988*
EchoNet-PediatricCardiacState4.725ref49.599ref
EchoNet-PediatricDINOv35.4309.999e-05***61.7240.00159984**
EchoNet-PediatricPanEcho4.8770.12848750.6960.39606
EchoNet-PediatricEchoPrime4.8070.31386940.3090.974303

* p < 0.05, ** p < 0.01, *** p < 0.001. P-values are unadjusted. Bold marks the lowest observed error for each metric and dataset.

At the nominal 0.05 level, CardiacState achieved lower MAE and MSE than all three baselines on EchoNet-Dynamic and than DINOv3 on EchoNet-Pediatric. On EchoNet-Pediatric, CardiacState had the lowest observed MAE. Its MAE gaps relative to PanEcho and EchoPrime were small and statistically inconclusive, at 0.152 and 0.082 percentage points, respectively. EchoPrime had the lowest pediatric MSE. The pediatric test set was more concentrated in a single LVEF band. 65.5% of its videos had a reference LVEF of 60% to less than 70%, compared with 39.6% on EchoNet-Dynamic. This gives that band greater weight in the aggregate metrics and may make differences elsewhere in the LVEF range harder to detect.

CardiacState learned these representations through self-supervised pretraining on 1.88 million echocardiography videos without LVEF labels or clinical reports. PanEcho trained on approximately 1.0 million videos with LVEF regression among its supervised targets. EchoPrime trained on approximately 12.0 million videos paired with cardiologist reports that contain LVEF measurements. Under the frozen-encoder evaluation, CardiacState’s adult test-set gains and competitive pediatric MAE show that self-supervised cardiac video pretraining can learn features that transfer to LVEF estimation without EF-specific supervision during pretraining.

We have also included the full pretraining computational cost in the revised manuscript:

Pretraining settingCardiacState
Trainable parameters1.18 billion1.18\text{ billion}
Training epochs1010
Hardware8×NVIDIA H200 GPUs8 \times \text{NVIDIA H200 GPUs}
Precisionbfloat16 mixed precision
Global batch size128 clips128\text{ clips}
Batch size per GPU16 clips16\text{ clips}
Wall-clock training time∼32.2 hours\sim 32.2\text{ hours}
Total compute∼257 H200 GPU-hours\sim 257\text{ H200 GPU-hours}
Peak GPU memory∼101 GiB per GPU\sim 101\text{ GiB per GPU}
Reviewer_SJR2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces CardiacState, a JEPA-based framework for learning cardiac representations from echocardiography videos. The key idea is to encode cardiac progression as a cyclic state on (S^1) and use a temporal bottleneck that forces future-feature prediction to condition on this state rather than directly accessing historical visual features. The method achieves improved LVEF estimation and cardiac keyframe localization across multiple benchmarks.

Critical strengths

1 - The use of a cyclic latent state is well motivated by the periodic nature of cardiac motion.

2 - The temporal bottleneck provides an interesting mechanism for encouraging structured temporal representations.

3 - The experiments cover multiple datasets and acoustic views. The ablations demonstrate the importance of the cardiac progression regularizer.

Critical weaknesses

1 - Limited evidence that the learned state captures physiological phase. The progression loss encourages forward angular movement but does not enforce one revolution per heartbeat or synchronization with the true cardiac cycle. A generic monotonically advancing circular representation could satisfy this objective. Quantitative analysis of phase alignment and cycle consistency is needed.

2 - Insufficient ablations to establish the main contribution. The paper does not compare against a standard JEPA trained under identical conditions without the cyclic state, or against an unconstrained low-dimensional temporal bottleneck. Consequently, it is unclear whether the improvements arise from the cyclic formulation or simply from compressing historical information.

3 - The role of the state in prediction is not sufficiently established. Although the bottleneck is designed to force reliance on the cardiac state, no intervention demonstrates that the predictor actually uses it. Shuffling or replacing the state during inference would provide direct evidence.

4 - Evaluation limitations. Keyframe localization uses the closest predicted event to each annotated frame, potentially favoring methods that generate excessive detections. Additionally, some published baselines use different protocols, and the OrbitalEcho comparison is available only for A4C. The LVEF improvements on EchoNet-Pediatric are relatively small, with EchoPrime achieving better MSE.

Questions for authors

1 - How well does the learned phase align with actual cardiac cycles? Does it consistently complete one revolution per heartbeat?

2 - How would an unconstrained 2D state or a simple temporal embedding perform under the same bottleneck?

3 - How much does future-feature prediction deteriorate when the cardiac state is shuffled or replaced with a constant?

4 - How does the approach handle different heart rates, particularly when large temporal sampling gaps cause angular wrap-around ambiguity?

5 - Can the authors report event-level precision and recall for keyframe localization rather than only nearest-prediction MAE?

9 comments

We thank Reviewer 1 for the constructive questions regarding physiological phase alignment, matched baselines, causal state perturbations, sampling-gap robustness, and detection counts.

1. Alignment with Physiological Cardiac Cycles & Single-Revolution Consistency (W1 & Q1)

We have moved Figure 2 from the appendix into the main paper to illustrate learned cardiac-state trajectories across an entire multi-beat video. Under the K=1K=1 bottleneck with Lcardiac\mathcal{L}_{\text{cardiac}}, the learned state moves smoothly around the unit circle, completing multiple revolutions over the example video.

Public echocardiography benchmarks provide sparse single-beat ED/ES annotations, and no publicly available dataset provides exhaustive multi-cycle keyframe annotations. To assess cycle consistency across all test videos, we estimate each video's heartbeat count by treating twice the annotated ED–ES interval as one cardiac cycle length, and divide the number of predicted ED and ES events by this estimated cycle count:

\text{detection density} = \frac{N_{\text{predictions}}}{\widehat{N}_{\text{cycles}}}.$$ One ED and one ES event per beat corresponds to an expected density of approximately $2.0$. Across all three views, our $K=1$ model averages **$2.280 \pm 0.934$** detections per estimated cycle, suggesting that the low localization error is not driven by dense over-prediction: | View | Method | MAE $\downarrow$ | RMSE $\downarrow$ | $R^2$ $\uparrow$ | Estimated cycles | Estimated cycle frame length | Estimated detection density | | :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | A4C | $K=4$, w/o $\mathcal{L}_{\text{cardiac}}$ | $28.475 \pm 0.997$ | $49.082 \pm 1.395$ | $-0.429 \pm 0.094$ | $5.50 \pm 3.13$ | $33.91 \pm 10.98$ | $2.217 \pm 1.979$ | | A4C | $K=4$, w/ $\mathcal{L}_{\text{cardiac}}$ | $2.105 \pm 0.029$ | $2.518 \pm 0.028$ | $0.9962 \pm 0.0002$ | $5.50 \pm 3.13$ | $33.91 \pm 10.98$ | $2.200 \pm 0.708$ | | A4C | $K=1$, w/ $\mathcal{L}_{\text{cardiac}}$ | **$1.985 \pm 0.024$** | **$2.325 \pm 0.022$** | **$0.9967 \pm 0.0002$** | $5.50 \pm 3.13$ | $33.91 \pm 10.98$ | **$2.137 \pm 0.692$** | | PSAX | $K=4$, w/o $\mathcal{L}_{\text{cardiac}}$ | $15.183 \pm 0.883$ | $27.059 \pm 1.690$ | $0.501 \pm 0.137$ | $2.67 \pm 1.67$ | $39.42 \pm 21.30$ | $2.724 \pm 2.971$ | | PSAX | $K=4$, w/ $\mathcal{L}_{\text{cardiac}}$ | $2.083 \pm 0.047$ | $2.573 \pm 0.051$ | $0.9953 \pm 0.0011$ | $2.67 \pm 1.67$ | $39.42 \pm 21.30$ | $2.546 \pm 1.370$ | | PSAX | $K=1$, w/ $\mathcal{L}_{\text{cardiac}}$ | **$2.027 \pm 0.045$** | **$2.452 \pm 0.054$** | **$0.9958 \pm 0.0010$** | $2.67 \pm 1.67$ | $39.42 \pm 21.30$ | **$2.518 \pm 1.332$** | | PLAX | $K=4$, w/o $\mathcal{L}_{\text{cardiac}}$ | $11.531 \pm 1.055$ | $24.956 \pm 2.077$ | $0.572 \pm 0.101$ | $5.20 \pm 1.83$ | $34.68 \pm 8.29$ | $3.213 \pm 1.558$ | | PLAX | $K=4$, w/ $\mathcal{L}_{\text{cardiac}}$ | $2.135 \pm 0.061$ | $2.597 \pm 0.058$ | $0.9954 \pm 0.0008$ | $5.20 \pm 1.83$ | $34.68 \pm 8.29$ | $2.327 \pm 0.562$ | | PLAX | $K=1$, w/ $\mathcal{L}_{\text{cardiac}}$ | **$1.994 \pm 0.049$** | **$2.358 \pm 0.046$** | **$0.9962 \pm 0.0006$** | $5.20 \pm 1.83$ | $34.68 \pm 8.29$ | **$2.185 \pm 0.528$** | | Avg | $K=4$, w/o $\mathcal{L}_{\text{cardiac}}$ | $18.396 \pm 0.570$ | $33.699 \pm 1.011$ | $0.215 \pm 0.065$ | $4.46 \pm 2.63$ | $36.01 \pm 14.84$ | $2.718 \pm 2.285$ | | Avg | $K=4$, w/ $\mathcal{L}_{\text{cardiac}}$ | $2.107 \pm 0.027$ | $2.563 \pm 0.028$ | $0.9956 \pm 0.0005$ | $4.46 \pm 2.63$ | $36.01 \pm 14.84$ | $2.357 \pm 0.958$ | | Avg | $K=1$, w/ $\mathcal{L}_{\text{cardiac}}$ | **$2.002 \pm 0.023$** | **$2.378 \pm 0.024$** | **$0.9962 \pm 0.0004$** | $4.46 \pm 2.63$ | $36.01 \pm 14.84$ | **$2.280 \pm 0.934$** |

2. Matched Standard JEPA Baseline & Bottleneck Ablations (W2 & Q2)

To isolate the contribution of explicit cyclic-state conditioning from standard feature prediction, we trained a matched time-conditioned K=4K=4 JEPA baseline without the cardiac-state module or Lcardiac\mathcal{L}_{\text{cardiac}} under identical initialization, pretraining data, sampling, and compute budget. Because this baseline produces no phase representation, keyframe localization is not applicable (N/A), and we evaluate its frozen visual representations on LVEF estimation alongside our ablation variants:

MethodEchoNet-Dynamic MAE ↓\downarrowEchoNet-Dynamic MSE ↓\downarrowEchoNet-Pediatric MAE ↓\downarrowEchoNet-Pediatric MSE ↓\downarrowLocalization Mean MAE (frames) ↓\downarrow
K=4K=4, JEPA w/o cardiac state5.304±0.2565.304 \pm 0.25652.131±2.32852.131 \pm 2.3285.041±0.2785.041 \pm 0.27857.614±7.03257.614 \pm 7.032N/A
K=4K=4, w/o Lcardiac\mathcal{L}_{\text{cardiac}}5.204±0.1335.204 \pm 0.13350.154±2.98050.154 \pm 2.9805.070±0.1915.070 \pm 0.19158.648±6.11058.648 \pm 6.11018.396±0.57018.396 \pm 0.570
K=4K=4, w/ Lcardiac\mathcal{L}_{\text{cardiac}}5.028±0.1245.028 \pm 0.12445.549±2.38545.549 \pm 2.3854.811±0.1754.811 \pm 0.17550.835±5.39750.835 \pm 5.3972.107±0.0272.107 \pm 0.027
K=1K=1, w/ Lcardiac\mathcal{L}_{\text{cardiac}}4.867±0.1214.867 \pm 0.12142.554±2.24942.554 \pm 2.2494.725±0.1754.725 \pm 0.17549.599±5.18849.599 \pm 5.1882.002±0.0232.002 \pm 0.023

When interpreting keyframe localization, the 18.39618.396-frame MAE of the K=4 w/o LcardiacK=4\text{ w/o }\mathcal{L}_{\text{cardiac}} variant should not be viewed as a functional localization baseline. Without Lcardiac\mathcal{L}_{\text{cardiac}}, the state module suffers from representation collapse—remaining nearly stationary around a narrow angle (Figure 2, top)—which makes the phase readout degenerate and functionally equivalent to having no usable state output (analogous to the N/A baseline above).

The meaningful external reference under the identical closest-prediction protocol on A4C is OrbitalEcho at 2.2452.245 frames MAE (Table 2). Relative to this 2.2452.245-frame benchmark, adding Lcardiac\mathcal{L}_{\text{cardiac}} at K=4K=4 reaches 2.1052.105 frames on A4C (−0.140-0.140 frames), and tightening the temporal bottleneck from K=4K=4 to K=1K=1 further reduces A4C error to 1.9851.985 frames (−0.120-0.120 frames, Tables 2 and 5)—accounting for 46.2% of our total localization improvement over OrbitalEcho.

3. Causal Intervention on the Inferred Cardiac State (W3 & Q3)

We tested the predictor's reliance on the cardiac state by intervening on ctc_t at inference. Specifically, we froze each pretrained model, rotated the current state ct∈S1c_t \in S^1 by a specified angle, and predicted the next-frame feature while keeping the visual input FtF_t and elapsed time Δτt\Delta\tau_t fixed. Below we report the feature RMSE between predictions made with the rotated state and those made with the original state across all three datasets:

EchoNet-Dynamic

RotationK=4K=4, w/o Lcardiac\mathcal{L}_{\text{cardiac}}K=4K=4, w/ Lcardiac\mathcal{L}_{\text{cardiac}}K=1K=1, w/ Lcardiac\mathcal{L}_{\text{cardiac}}
10∘10^\circ1.680±0.0871.680 \pm 0.0871.674±0.0951.674 \pm 0.0952.162±0.1212.162 \pm 0.121
20∘20^\circ1.735±0.0921.735 \pm 0.0921.740±0.0961.740 \pm 0.0962.482±0.1402.482 \pm 0.140
30∘30^\circ1.770±0.0951.770 \pm 0.0951.793±0.0981.793 \pm 0.0982.781±0.1612.781 \pm 0.161
60∘60^\circ1.844±0.0981.844 \pm 0.0981.889±0.1021.889 \pm 0.1023.822±0.2463.822 \pm 0.246
90∘90^\circ1.898±0.1021.898 \pm 0.1021.948±0.1051.948 \pm 0.1054.769±0.4134.769 \pm 0.413
120∘120^\circ1.943±0.1081.943 \pm 0.1081.962±0.1051.962 \pm 0.1055.201±0.6565.201 \pm 0.656

EchoNet-Pediatric

RotationK=4K=4, w/o Lcardiac\mathcal{L}_{\text{cardiac}}K=4K=4, w/ Lcardiac\mathcal{L}_{\text{cardiac}}K=1K=1, w/ Lcardiac\mathcal{L}_{\text{cardiac}}
10∘10^\circ1.725±0.1141.725 \pm 0.1141.862±0.1231.862 \pm 0.1232.409±0.1402.409 \pm 0.140
20∘20^\circ1.766±0.1111.766 \pm 0.1111.936±0.1251.936 \pm 0.1252.781±0.1632.781 \pm 0.163
30∘30^\circ1.800±0.1181.800 \pm 0.1181.987±0.1251.987 \pm 0.1253.123±0.1933.123 \pm 0.193
60∘60^\circ1.873±0.1211.873 \pm 0.1212.097±0.1292.097 \pm 0.1294.242±0.3934.242 \pm 0.393
90∘90^\circ1.928±0.1311.928 \pm 0.1312.159±0.1342.159 \pm 0.1345.097±0.7165.097 \pm 0.716
120∘120^\circ1.968±0.1341.968 \pm 0.1342.173±0.1332.173 \pm 0.1335.311±0.9855.311 \pm 0.985

EchoNet-LVH

RotationK=4K=4, w/o Lcardiac\mathcal{L}_{\text{cardiac}}K=4K=4, w/ Lcardiac\mathcal{L}_{\text{cardiac}}K=1K=1, w/ Lcardiac\mathcal{L}_{\text{cardiac}}
10∘10^\circ1.473±0.1041.473 \pm 0.1041.590±0.1141.590 \pm 0.1141.919±0.1571.919 \pm 0.157
20∘20^\circ1.508±0.1071.508 \pm 0.1071.652±0.1131.652 \pm 0.1132.202±0.1792.202 \pm 0.179
30∘30^\circ1.527±0.1131.527 \pm 0.1131.707±0.1151.707 \pm 0.1152.462±0.2312.462 \pm 0.231
60∘60^\circ1.595±0.1161.595 \pm 0.1161.785±0.1261.785 \pm 0.1263.231±0.5223.231 \pm 0.522
90∘90^\circ1.638±0.1201.638 \pm 0.1201.836±0.1281.836 \pm 0.1283.698±0.8893.698 \pm 0.889
120∘120^\circ1.679±0.1201.679 \pm 0.1201.854±0.1271.854 \pm 0.1273.748±1.1793.748 \pm 1.179

Across all three datasets, larger state rotations produce progressively larger shifts in predicted features, with consistently greater sensitivity at K=1K=1 than at K=4K=4 (e.g., 4.7694.769 vs. 1.9481.948 at 90∘90^\circ on EchoNet-Dynamic). This directly confirms that the tighter visual bottleneck increases the predictor's reliance on the inferred cardiac state.

4. Robustness to Heart-Rate Variations and Large Sampling Gaps (Q4)

CardiacState handles heart-rate and frame-rate variations by conditioning on physical elapsed time Δτt\Delta\tau_t (in ms), pretraining across multiple strides (s∈{2,3,4,5,6}s \in \{2, 3, 4, 5, 6\}), and enforcing a velocity-flexible progression lower bound (ϵ=10−3 rad/ms\epsilon = 10^{-3}\text{ rad/ms}) rather than a fixed angular speed. Consequently, localization remains stable (∼2.0\sim 2.0 frames MAE) across distinct acquisition and heart-rate regimes, including EchoNet-LVH (∼20 FPS\sim 20\text{ FPS}, 1.9941.994), EchoNet-Dynamic (∼50 FPS\sim 50\text{ FPS}, 1.9851.985), and faster pediatric cohorts (>100 FPS>100\text{ FPS}, 2.0272.027).

To test sparser temporal sampling, we re-ran ED/ES localization on the same videos at gaps of 22, 33, and 44 frames:

Sampling gap (frames)ED–ES sampling steps, median [Q1,Q3][Q_1, Q_3]Localization Mean MAE (frames) ↓\downarrow
228.00 [7.00,9.50]8.00\ [7.00, 9.50]2.002±0.0232.002 \pm 0.023
335.33 [4.67,6.33]5.33\ [4.67, 6.33]2.451±0.3362.451 \pm 0.336
444.00 [3.50,4.75]4.00\ [3.50, 4.75]3.237±1.0793.237 \pm 1.079

Degradation at wider gaps reflects a fundamental aliasing limit of periodic signals rather than a model flaw: cardiac motion is cyclic, so overly large sampling intervals destroy intermediate trajectory history and create false apparent dynamics—just as sampling a sine wave near half-period or period multiples aligns discrete points along a flat straight line (appearing stationary) or induces apparent phase reversal. Standard clinical frame rates (≥8\ge 8 steps per half-cycle at stride 22) remain well above this aliasing threshold.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.