- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Health-Aware Manifold Learning (HAML) for remaining useful life (RUL) prediction. It first fits a representation without lifecycle labels, then introduces lifecycle supervision through a progress-aware landmark graph and distributed healthy/failure reference regions. Relative graph-geodesic distances define a health coordinate, which a persistent level–velocity filter converts into a causal, bounded, non-decreasing health index (HI). A downstream decoder combines HI history, temporal features, and complementary geometric information. Experiments on C-MAPSS and PHM12 assess predictive performance, health-trajectory diagnostics, robustness, and representation-backend portability.
Critical strengths
1.A useful and explicit formulation of health-state construction. HAML separates representation learning, supervised health geometry, causal trajectory construction, and RUL decoding. Distributed endpoint references accommodate heterogeneous degradation states, while the filter imposes clearly specified online trajectory constraints. The paper appropriately limits its theoretical claims and explicitly acknowledges that these constraints do not guarantee predictive optimality or physical validity.
2.Substantial experimental coverage and informative controls. The evaluation includes two benchmark families, ten-seed reporting, noise perturbations, eight ablation groups, and multiple representation backends. Figure G1 compares the original descriptors, UMAP coordinates, and HAML features across several decoders; Table G1 additionally supplies descriptor-matched neural baselines. These are valuable controls. The authors also report unfavorable results, including cases where direct descriptor-based prediction is stronger and where removing filtering improves accuracy.
Critical weaknesses
1.The population and implementation used for the HI diagnostics are insufficiently specified, and some reported results are difficult to reconcile with their definitions. Appendix D states that PHM12 Condition 3 has only one test bearing, whereas Eq. D7 defines prognosability using the standard deviation of failure-endpoint HI values across assets. Table G5 nevertheless reports 0.996 ± 0.003 for this condition. If the metric is computed separately for each seed on that single held-out bearing, the population standard deviation is zero, giving prognosability 1 when the denominator is nonzero; the sample standard deviation would be undefined. The evaluated bearing population and aggregation across assets and seeds therefore need to be stated. In addition, Eq. D5 uses signed increments: under the usual convention sgn(0) = 0, a non-decreasing trajectory containing plateaus need not receive a score of 1, contrary to the accompanying construction-based guarantee. These observations do not establish an implementation error, but they prevent a clear interpretation of evidence central to the paper’s health-state claims.
2.The additional value of the constrained health state is not sufficiently separated from imposed temporal structure. In Table 2, removing filtering (A5) improves the reported mean RMSE, S-score, and P95 Late Error on all four C-MAPSS subsets. For example, RMSE changes from 13.13 to 12.77 on FD001 and from 14.31 to 13.73 on FD003. The authors explicitly acknowledge this accuracy–constraint trade-off, which is reasonable. However, the compensating health-state benefit needs stronger validation. In Eq. D6, normalized lifecycle progress is an affine transformation of time within each asset, so the reported per-asset Pearson correlation is also correlation with time. This measures temporal alignment but does not by itself establish additional sensor-derived degradation information. Furthermore, replacing the filter with cumulative maximum in A6 changes several mechanisms simultaneously, so it does not cleanly isolate persistent velocity memory. The full-pipeline noise experiments likewise do not isolate the filter’s contribution. An age-only diagnostic control and a comparison retaining the smoothing and projection rules while removing persistent velocity would help establish the specific benefit.
Questions for authors
1.Which exact assets and trajectory segments enter the HI diagnostics in Tables G2 and G5, and are these assets excluded from representation fitting, graph construction, and reference selection? How are statistics aggregated across assets and seeds, particularly for PHM12 Condition 3?
2.How are zero increments handled in the implementation of Eq. D5? Does the reported metric implement the displayed sign-based expression or a non-negative-increment proportion?
3.What evidence shows that the persistent level–velocity mechanism provides useful health-state information beyond age alignment and generic monotone smoothing? In particular, is its claimed benefit retained when smoothing and projection are held fixed, or can the existing noise evaluation distinguish the full filter from A5 and a matched simpler filter?
6 comments
Response to reviewer oSrf (part 2)
3. Eq. D5 and zero increments (Q2, W1). We agree: with , the displayed sign-based formula is inconsistent with counting plateaus as non-decreasing steps. The metric described by our implementation is the fraction of adjacent pairs satisfying , pooled across eligible training-asset pairs within each seed and reported as mean ± SD across ten seeds. We have verified that the reported values were computed using the indicator-based definition and remain unchanged. We will replace Eq. D5 accordingly and clarify the numerical tolerance and aggregation procedure. A monotonicity score of 1.000 verifies the imposed projection (Eq. 13); it is not evidence of physical degradation validity.
4. Velocity persistence versus smoothing (Q3, W2). Removing filtering (A5) improves clean-data prediction but sacrifices monotonicity; cumulative maximum (A6) alters multiple mechanisms and cannot isolate velocity persistence. We therefore reran matched A0, V1 (; no carried velocity) and V2 (, ; no velocity contribution) over the same ten seeds, preserving graph features, smoothing/projection rules and 49-dimensional decoder inputs. All variants remain monotone.
A0 yields higher training-lifecycle prognosability (0.984–0.999 vs. 0.965–0.984) and healthy-failure separation (0.809–0.914 versus 0.767–0.870). Exact paired sign-flip tests with Holm correction across 24 HI comparisons (3 metrics × 2 contrasts × 4 datasets) yield a minimum Holm-adjusted p-value of 0.0469. Predictive effects are smaller: none of the 24 paired RUL comparisons remains significant after Holm correction (minimum adjusted p = 0.281). Some late-error outcomes favour the ablations (Table R3). Thus, the full filter improves training-lifecycle HI diagnostics but does not demonstrate statistically significant RUL superiority over the matched controls. The existing noise experiment does not isolate the filter's contribution. Detailed statistical results will be provided in the appendix. The existing noise experiment does not isolate the filter (Table R3).
Table R3. Velocity-component ablations: training-lifecycle HI diagnostics and held-out RUL prediction.
| Dataset | Variant | HI–progress r ↑ | Prognosability ↑ | Healthy–failure separation ↑ | RMSE ↓ | NASA S-score ↓ | P95 late error ↓ |
|---|---|---|---|---|---|---|---|
| FD001 | A0 | 0.903 ± 0.007 | 0.998 ± 0.004 | 0.914 ± 0.010 | 13.11 ± 0.50 | 270.9 ± 41.5 | 24.40 ± 3.07 |
| V1 | 0.901 ± 0.006 | 0.984 ± 0.005 | 0.870 ± 0.009 | 13.23 ± 0.64 | 277.3 ± 49.6 | 25.13 ± 3.73 | |
| V2 | 0.901 ± 0.006 | 0.984 ± 0.005 | 0.870 ± 0.009 | 13.24 ± 0.68 | 277.3 ± 51.0 | 25.08 ± 3.52 | |
| FD002 | A0 | 0.893 ± 0.007 | 0.998 ± 0.002 | 0.897 ± 0.009 | 14.86 ± 0.63 | 891.3 ± 157.7 | 29.92 ± 2.36 |
| V1 | 0.892 ± 0.007 | 0.980 ± 0.005 | 0.844 ± 0.010 | 15.04 ± 0.57 | 937.2 ± 147.6 | 30.07 ± 1.99 | |
| V2 | 0.892 ± 0.007 | 0.980 ± 0.005 | 0.844 ± 0.010 | 15.01 ± 0.52 | 924.9 ± 128.8 | 30.03 ± 2.14 | |
| FD003 | A0 | 0.872 ± 0.009 | 0.999 ± 0.002 | 0.890 ± 0.007 | 14.39 ± 0.70 | 922.3 ± 311.2 | 33.13 ± 1.86 |
| V1 | 0.870 ± 0.008 | 0.983 ± 0.005 | 0.856 ± 0.008 | 14.41 ± 0.58 | 936.4 ± 266.4 | 32.61 ± 1.56 | |
| V2 | 0.870 ± 0.008 | 0.983 ± 0.005 | 0.856 ± 0.008 | 14.41 ± 0.59 | 949.0 ± 277.5 | 32.61 ± 1.58 | |
| FD004 | A0 | 0.859 ± 0.013 | 0.984 ± 0.017 | 0.809 ± 0.085 | 15.85 ± 0.34 | 1090.3 ± 78.0 | 28.05 ± 2.78 |
| V1 | 0.858 ± 0.013 | 0.965 ± 0.012 | 0.767 ± 0.081 | 15.85 ± 0.38 | 1087.0 ± 103.6 | 27.75 ± 1.84 | |
| V2 | 0.858 ± 0.013 | 0.965 ± 0.012 | 0.767 ± 0.081 | 15.88 ± 0.43 | 1095.6 ± 114.0 | 28.02 ± 1.83 |
A0 = full HAML rerun for this experiment; V1 = no velocity persistence (); V2 = no velocity contribution (, ). Results are mean ± SD over ten matched seeds. HI diagnostics are computed on complete training trajectories, whereas RUL metrics are evaluated on official held-out test assets. All configurations retain smoothing and projection and achieve monotonicity of 1.000. A0 was independently rerun alongside V1/V2 under the matched experimental protocol. The resulting A0 values are reported separately from the original manuscript results to ensure within-experiment comparability. Lower RMSE, NASA S-score, and P95 Late Error are better; higher HI correlation, prognosability, and separation are better.
Follow-up on HI Evaluation and Remaining Concerns
Thank you for the clarifications and additional experiments. The released code supports your explanation of the implemented monotonicity metric and the use of training trajectories for the original HI diagnostics. I appreciate the engine-disjoint age-only comparisons and the matched velocity ablations, which address important parts of my concerns.
However, the remaining evidence is not yet sufficient to establish the additional value of the proposed health-state construction. In Table R2, terminal HI saturation limits the interpretation of prognosability, while the unmatched decoder capacity prevents attributing the RUL advantage specifically to the HI. Table R3 provides a better-controlled comparison, but its HI improvements are measured on training trajectories, and no RUL comparison remains significant after Holm correction. This does not establish that the velocity component is ineffective, but its practical benefit remains insufficiently demonstrated.
One implementation detail also needs clarification: Eqs. D6 and D9 specify per-engine calculations followed by averaging, whereas hi_metrics in the released metrics.py computes correlation and separation over pooled observations. These procedures are generally not equivalent. Please clarify which aggregation was used for the original results and Tables R2–R3, and reconcile the definitions with the corresponding implementation.
I acknowledge the improvements in the rebuttal, but maintain my current rating for now because the central evidential concerns remain only partially resolved.
Response to reviewer oSrf (part 1)
We thank the reviewer for their constructive feedback on our methodology, particularly on HI evaluation and the distinction between imposed temporal constraints and sensor-derived degradation information. To address these concerns, we conducted additional comparisons against age-only baselines and matched velocity-filter ablations (Tables R2–R3). These results provide further evidence that the learned health representations capture sensor-derived information beyond elapsed age, and that velocity persistence improves HI diagnostics across the training lifecycle. We address each point below.
1. HI evaluation population and aggregation (Q1, W1). Tables G2/G5 and Fig. 3/Table G6 use complete run-to-failure training trajectories: C-MAPSS FD001–FD004 engines from their first valid 50-cycle-window endpoint to failure, and PHM12 bearings 1_1/1_2, 2_1/2_2 and 3_1/3_2. These assets also enter representation fitting, graph construction, and healthy/failure reference selection. Thus, the diagnostics describe in-sample lifecycle structure and constraint compliance, not held-out HI generalisation. We will correct the misleading “held-out lifecycle progress” wording in Section 4.4. Held-out RUL evaluation remains separate and uses no future test observations or failure endpoints. This distinction follows the endpoint-dependent definition of prognosability in Coble and Hines [1]; Dama and Sinoquet [2] provide a C-MAPSS precedent for health-state construction on complete training histories with separate RUL evaluation on test histories.
For PHM12 Condition 3, Eq. D7 is evaluated within each seed on training bearings 3_1 and 3_2, not the single test bearing 3_3. The numerator is the population SD of their terminal HI values; the denominator is their mean absolute start-to-end HI change. The result 0.996 ± 0.003 is the mean ± SD over ten seed-level scores, not uncertainty over one test bearing. We will state the asset sets and aggregation rules explicitly, and acknowledge that two training bearings provide limited evidence of population-level prognosability.
References
[1] Coble, J., & Hines, J. W. (2009). Identifying optimal prognostic parameters from data: A genetic algorithms approach. Annual Conference of the PHM Society, 1(1).
[2] Dama, F., & Sinoquet, C. (2023). Partially Hidden Markov Chain Multivariate Linear Autoregressive model: Inference and forecasting—application to machine health prognostics. Machine Learning, 112, 45–97.
2. Beyond age alignment (W2). HI-progress correlation alone does not demonstrate sensor-derived degradation information. We therefore compared HAML with linear (Age-L) HI(t)=min(max(T−50t−50,0),1) and isotonic (Age-NL)HI(t)=fiso(t) age-only controls using five-fold, engine-disjoint evaluation over ten split seeds across FD001-FD004. Preprocessing, UMAP, graph fitting, and filter calibration used calibration engines only; HI diagnostics used held-out complete trajectories. HAML achieved higher healthy–failure separation than Age-L (0.887–0.930 vs 0.843–0.881) and prognosability than Age-NL (0.984–0.997 vs 0.897–0.920), although Age-L had higher progress correlation (0.976–0.990 vs 0.896–0.925). On official test engines, full HAML outperformed Age-NL in RUL prediction (FD003 RMSE 14.31 vs 31.84). These results support sensor-informed information beyond age, although terminal HI saturation (94–98%) and unmatched decoder capacity limit attribution to HI alone (Table R2).
Table R2. Engine-disjoint evaluation of HAML versus age-only controls: held-out HI diagnostics and test-set RUL prediction.
HI diagnostics are evaluated on held-out complete run-to-failure engines using five-fold, engine-disjoint validation repeated over ten splits. Age-L denotes linear age, and Age-NL denotes isotonic nonlinear age. The displayed comparators are Age-L for Pearson correlation and separation, and Age-NL for prognosability and RUL prediction. RUL results use official held-out test engines; no age-only predictor uses an individual test engine's failure time. The RUL comparison is not decoder-capacity-matched. HAML terminal HI saturation may inflate prognosability.