acceptodds
Under review as a conference paper at ICLR 2027

Condition-Specific Weighted Data Sparsification for Compute-Efficient Physical-System Regression

Abstract

Industrial physical-system models are often retrained from dense datasets collected across various operating conditions. Full-data retraining can incur substantial cloud-side computation to process redundant observations, whereas aggressive pruning may discard rare or difficult states. This work introduces a condition-specific data sparsification framework for cost-constrained industrial machine learning on physical-system regression tasks. The framework formulates sparsification as a condition-specific weighted empirical approximation. Geometric representatives define the retained support, assignment counts preserve the empirical mass of omitted observations, and residual feedback corrects task-relevant omissions under a condition-specific budget. The procedure operates at the training-support level, leaving the predictor architectures and forward computations unchanged. The framework is evaluated on battery state-of-charge (SOC) estimation and gas-turbine NO prediction using paired model-initialization seeds. The reported analytical cloud-side FLOP counts cover the modeled data-selection, retraining, and validation computations. The final retention ratios are 39.2%–41.4% for SOC and 38.7%–41.7% for NO. In the retention-matched battery MLP experiments, the framework obtains lower mean MAE than Random in two of four driving cycles and than GradMatch in three of four cycles, while using 10.1%–16.2% fewer mean analytical cloud-side FLOPs than GradMatch. Across all settings, seed-averaged FLOP reductions are 8.3%–89.2% for SOC and 20.0%–85.1% for NO, while setting-level, seed-averaged relative MAE changes range from -9.8% to +18.0% and from -0.9% to +13.4%, respectively. These results characterize when condition-specific data sparsification is beneficial and support the selection of retention budgets for cost-constrained industrial retraining.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
industrial machine learning, condition-specific data sparsification, physical-system regression, residual feedback, retention budgets

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_jJogOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies data sparsification for repeated training of industrial regression models. It partitions measurements by operating condition and constructs an initial support using clustering and geometric criteria. Retained samples receive frequency weights based on how many omitted observations they represent. A predictor is trained on this weighted support, with optional full-data warm-up. Its residuals on omitted training samples identify difficult examples for re-entry. Representative assignments and weights are then recomputed before a final training stage, while the predictor architecture remains unchanged. The goal is to reduce training computation while preserving coverage and regression accuracy.

Critical strengths
  1. The paper addresses a practical problem, reducing retraining costs without changing the deployed predictor. This makes the approach relatively easy to integrate into existing regression pipelines.
  2. The representative-frequency weighting provides an explicit accounting of the observations represented by each retained sample. The condition-specific construction also offers a straightforward way to maintain coverage across operating regimes.
  3. The experiments use paired initialization seeds and define temporal splits before constructing windows. Reporting per-seed results and unfavorable accuracy outcomes makes the empirical trade-offs reasonably transparent.
Critical weaknesses
  1. The feedback score in Eq. 25 produces exactly the same within-condition ranking as absolute residual alone, because Huber loss is monotone in residual magnitude. More broadly, the methodological advance beyond combining clustering, representative weighting, and hard-example selection appears limited.
  2. The computational accounting excludes full-pool feedback inference, representative reassignment, and weight recomputation, despite these being required pipeline operations. Without complete accounting or end-to-end runtime measurements, the reported fLOP reductions do not establish practical acceleration.
  3. The retention-matched baseline comparison is limited to battery MLPs, where the method beats Random in only two of four cycles. Its advantage over simpler selection methods is therefore not established for the other architectures or the NOx task.
  4. Comparisons between the non-re-entry and re-entry configurations change retention and topology handling alongside feedback, with some variants also differing in optimizer or checkpoint rules. These comparisons do not isolate the benefit of residual-guided restoration under otherwise identical conditions.
  5. Preserving total sample mass does not establish that the weighted support closely approximates the full regression objective. The paper provides neither an approximation-error diagnostic nor targeted rare-regime results showing that input-space representatives preserve task-relevant variation.
Questions for authors

See Weakness.

2 comments
AuthorsOpenReview ↗

Response to Reviewer jJog

We thank Reviewer jJog for the precise technical comments. The revision addresses each point with explicit corrections and narrower claims.

W1 — Eq. 25 ranking and novelty. The observation is correct. Since both terms in Eq. 25 are monotone in |e|, their positive convex combination gives the same within-condition ranking as |e| alone, for any alpha in (0,1), including the fixed alpha=0.5. Numerical checks confirmed identical re-entry index sets. The revised text now states this explicitly, clarifies that alpha is not tuned, and removes any claim of extra discriminative power. The two-term form is retained only for tie robustness. We also accept that clustering, frequency weighting, and hard-example mining are established; the contribution is now scoped to their condition-partitioned, mass-preserving composition with residual-guided re-entry.

W2 — complete accounting and runtime. We accept the accounting concern. The revision includes full-pool feedback inference, cKDTree reassignment, post-re-entry weight recomputation, topology/landmark scoring, re-entry validation, all training and validation passes, and optimizer updates, with the same scope for all methods. Under this scope, MLP battery delta-FLOPs are -48.8% on HWFET, -11.2% on LA92, -34.9% on UDDS, and -7.2% on US06; overall reductions are 7.2%-87.2% for SOC and 19.9%-84.8% for NOx. Appendix H reports direct wall-clock for all 21 settings: the proposed arm is faster than Full in 19, faster than GradMatch in 21, and within +/-7% of Random in 20. For US06 MLP, median times are 482 s Full, 667 s Random, 692 s GradMatch, and 642 s Ours; the proposed arm is slower than Full there but faster than both baselines. The claim is now an accuracy-compute trade-off, not uniform speedup.

W3 — baseline breadth. The comparison now covers 3 architectures across 4 battery cycles and 3 gas-turbine years, giving 21 cells with three paired seeds each. On battery, the proposed method has lower mean MAE than Random in 8 of 12 settings; the remaining differences are unfavorable or within seed-level noise. On gas turbine, no accuracy win is claimed: MLP saves about 76%-85% FLOPs at up to 6.7% MAE cost, while FCN/CNN costs reach 13.4% near 40% retention. Against GradMatch, lower mean MAE is obtained in 7 of 12 battery settings. These cases are now stated in the revised claims and Limitations.

W4 — isolating residual-guided re-entry. A controlled random re-entry arm has been added with initial support, retention budget, optimizer state, checkpoint rule, and topology fixed; only candidate ranking differs. The number of re-entered samples is matched per cycle: UDDS 1987, HWFET 3263, LA92 1352, US06 5618. Residual-guided re-entry lowers MAE in 8 of 12 paired runs; random re-entry increases mean MAE by +4.7%, with largest gaps on UDDS (+12.0%) and LA92 (+7.2%). This isolates candidate ranking under otherwise identical conditions.

W5 — approximation and rare regimes. Appendix I adds diagnostics. Full-pool unweighted MSE is +4.4% to +14.9% for the proposed model, versus -1% to +6.5% for Random. In the hardest target-quantile bin, proposed-vs-Full MAE is -11.1% US06, -5.4% LA92, -1.7% UDDS, and +0.7% HWFET. The opposite extreme bins are also disclosed: +11.7% US06, +13.6% UDDS, +4.8% HWFET, and -1.9% LA92. These bins contain few samples, and the degradation is stated as a limitation. The revised text no longer implies that mass preservation guarantees close approximation of the full objective.

All corrections, expanded comparisons, accounting changes, wall-clock results, controlled re-entry test, and diagnostics are reflected in the revised manuscript.

Thanks to authors, the rebuttal address most of my concerns.

Reviewer_u8UwOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The work proposes an approach for data sparsification to improve compute-efficiency in physical-system regression. The proposed scheme has three stages: 1) short training on the whole dataset, 2) training continuation on selected subset, 3) training continuation on selected subset plus the most difficult non-selected datapoints. The subset selection is again based on three steps, which are preceded by a K-means clustering: 1) Datapoints within a distance range to its cluster center are selected. 2) The closest datapoints to each cluster center are selected. 3) For time-series data, data with high changes compared to the preceding datapoint are selected. Additionally, datapoints with unretained landmarks are added to the selected subset. The approach is experimentally compared to random selection and to GradMatch on two datasets, battery SOC estimation and gas-turbine NOxNO_x prediction. The proposed method outperforms random selection in two out of four scenarios for SOC estimation and GradMatch in three out of four scenarios. Overall, the proposed approach reduces cloud-side FLOPs by 8.3%8.3\% - 89.2%89.2\% in the experiments compared to full training.

Critical strengths
  • The approach targets an important task of reducing redundant data in a training pipeline. The three staged approach of first shortly train with the whole dataset, then reduce the dataset for training continuation and finally add the most difficult datapoints again is convincing.
Critical weaknesses
  • Core concepts of subset selection are not well motivated. For example, the choice of bounds for Sdist,c\mathcal{S}_{dist,c} or Strans,c\mathcal{S}_{trans,c} are not explained. Additionally, theoretical results on the expected size of the initial support are missing.
  • Experimental results (Table 2) indicate that random selection of datapoints works comparably well as the proposed method. Since it should need even less FLOPs, I do not see that the experimental results support the overhead of the propopsed method.
Questions for authors

Clarification questions:

  • Sdist,c\mathcal{S}_{dist,c}: What is the motivation for the choice of the lower bound τcκic−\tau^-_{c{\kappa^c_i}} and the upper bound τcκic+\tau^+_{c{\kappa^c_i}}? Is it possible to quantify the amount of datapoints that you expect to be in Sdist,c\mathcal{S}_{dist,c}?
  • How is the elbow rank in line 204 defined? I am not sure what 'resulting curve' (l. 205) refers to.
  • Strans,c\mathcal{S}_{trans,c}: What is the motivation of comparing the ii-th datapoint at the last timestep with the i−1i-1-th datapoint at the last timestep? Could you elaborate on the motivation for the transition score and the choice of lower bound here?
  • How exactly does the auxiliary reference set work? Where do the landmarks come from and what do they represent?

Methodological questions:

  • I assume that the 'Condition-Specific Geometric Support' is the main methodological novelty, while 'Representative-Frequency Weighting' and 'Prediction-Error Feedback Correction' are well-known approaches. Is that correct?
  • Is there any way to control the size of Sgeo\mathcal{S}_{geo} or S0\mathcal{S}_0? How do you control the sparsification ratio e.g. for the experiments in Table 4?
1 comment
AuthorsOpenReview ↗

Response to Reviewer u8Uw

We thank Reviewer u8Uw for the careful and specific questions. Several math symbols were lost in the PDF rendering, so we answer each question in words and have also fixed the notation in the revision.

On the main novelty (methodological Q1). Partly correct. The individual ingredients — clustering, frequency weighting, hard-example mining — are established. What we claim is the composition under a condition-partitioned pipeline with an explicit mass-preservation constraint: (i) selection is performed within operating conditions so rare regimes cannot be pruned by a global criterion; (ii) retained samples carry weights that preserve total sample mass rather than being treated as i.i.d.; and (iii) re-entry re-selects omitted samples using the residual signal of the weighted model. We have revised the contribution statement to claim exactly this and no more, and we agree that the individual components are not novel.

On the distance and transition bounds (weakness 1). The bounds are data-derived quantile rules, not free hyperparameters. Within each cluster, centroid distances are sorted and smoothed with an odd moving window of width 2floor(n_k/20)+1 (minimum 3); the lower bound tau_minus is the distance at the "elbow" rank (see below). The upper bound is robust: tau_plus = median(d) + 31.4826*MAD(d), and is treated as unbounded when MAD = 0. The same multiplier s = 3 is used for the transition score, with MAD = 0 reducing the threshold to the median. The derivation has been added to Section 3.2.1 of the revised manuscript. We do not have a distribution-free finite-sample guarantee for the expected support size; we state that as an open theoretical gap rather than implying one exists.

Elbow rank and "resulting curve" (clarification). For a cluster of size n_k, sorted centroid distances are smoothed by that moving window, and both the rank axis and the smoothed-distance axis are normalized to [0,1]. The "resulting curve" is the plot of normalized smoothed distance against normalized rank. The elbow rank is the point of maximum absolute deviation of this curve from the unit diagonal — a standard knee-detection rule. It is therefore determined by the cluster's own distance distribution.

Transition score and its lower bound (clarification). The score compares the last timestep of window i with the last timestep of the immediately preceding window i-1 within the same contiguous segment; the first window of each segment and all non-sequential samples get zero. Comparing corresponding boundary timesteps of adjacent windows isolates the jump at the segment boundary, which is where a regime transition is visible, rather than the within-window trend. A sample is a transition representative when its score exceeds median(a) + 31.4826MAD(a) and its centroid distance is within tau_plus; the same robust multiplier keeps the rule scale-free across conditions.

Auxiliary reference set and landmarks (clarification). For each condition, up to 8 landmarks are chosen by farthest-point sampling in a standardized input-plus-target embedding. Their H0-persistence (lifetime in the minimum spanning tree of the landmarks) ranks their topological importance, and the most persistent ones are designated critical landmarks. In the main configuration topology protection is disabled, so landmarks are not inserted into the support; they only fix how many ordinary budget-matched replacements are added, prioritizing clusters with unretained landmarks, so the total retention budget stays matched. Landmarks thus represent extreme/topologically persistent points used to allocate replacements, not extra training samples.

Random works comparably, so why the overhead? (weakness 2). This is a fair point and we have adjusted the claim. Under complete accounting the proposed method costs about 6.8% more total FLOPs than retention-matched baselines at the same retention, and it lowers MAE in 8 of 12 battery seed runs — a measurable but not uniform gain. We now present this as a task- and architecture-dependent accuracy–compute trade-off rather than dominance, and we report the FLOPs overhead explicitly instead of leaving it implicit.

Controlling support size and the sparsification ratio (methodological Q2). The geometric thresholds above determine which samples qualify, but the final retention is set by an explicit target budget: the initial support is trimmed to the requested keep ratio, and Table 4 varies that target directly (e.g., 20/40/60/80%). So the reported sparsification ratios in Table 4 are controlled by this budget parameter, not left emergent from the thresholds.

All clarifications and the added derivation are in the revised manuscript. We thank the reviewer; these questions improved the precision of the paper.

Reviewer_Xn9DOpenReview ↗
Rating
2
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a condition-specific data sparsification method for physical-system regression. The method has three main elements: geometric support selection within each operating condition, representative-frequency weighting to preserve sample mass, and residual-based re-entry of difficult omitted samples. The model architecture is unchanged.

The method is evaluated on battery state-of-charge estimation and gas-turbine NOx prediction using MLP, FCN, and CNN models. The experiments use three paired initialisation seeds. The final retained datasets contain roughly 39% to 42% of the original training samples. The paper reports lower analytical FLOP counts in all evaluated settings, but accuracy changes substantially across datasets, conditions, and architectures.

The main comparison against Random and GradMatch is limited to the battery MLP experiments. The proposed method has lower mean MAE than Random on two of four cycles and lower mean MAE than GradMatch on three of four cycles.

Critical strengths

The method addresses a relevant industrial problem. It explicitly considers heterogeneous operating conditions rather than treating the training data as a single uniform pool. This is important for physical systems in which rare or difficult conditions may be lost through global pruning.

The experimental reporting is unusually detailed. The paper provides per-seed results, retention-budget analyses, architecture-specific results, training budgets, checkpoint rules, and explicit limits of its FLOP accounting. This improves transparency and supports reproduction.

Critical weaknesses

The compute-efficiency claim is not supported by complete cost measurements. The analysis excludes the full-dataset feedback pass and several method-specific data-processing operations. It also provides no wall-clock time, GPU time, energy use, or cloud cost. This is consequential because compute efficiency is the paper’s main motivation and one of its main claimed contributions. A method can reduce counted training FLOPs while increasing total runtime or system cost.

The empirical comparison is too narrow to establish a strong data-sparsification contribution. The full retention-matched comparison against Random and GradMatch is performed only for the battery MLP setting. The paper discusses CCS, CRAIG, BWS, and SELCON, but does not compare against them experimentally. The proposed method is not consistently better than Random, and its accuracy often decreases for the FCN and CNN gas-turbine settings. The current evidence does not show that the method generalises beyond a limited subset of the reported configurations.

Questions for authors

Clarification affecting the assessment: What are the total wall-clock training times and measured accelerator costs for Full Data, Random, GradMatch, and the proposed method? Please include clustering, the full-pool feedback pass, reassignment, weight recomputation, validation, and all training stages. Would the main compute-efficiency conclusion remain unchanged under this complete accounting?

Clarification affecting the assessment: Why is the retention-matched baseline study restricted to the battery MLP setting? Please report at least Random and GradMatch comparisons across the remaining architectures and the gas-turbine task, or explain why those comparisons cannot be performed.

2 comments
AuthorsOpenReview ↗

Response to Reviewer Xn9D

We thank Reviewer Xn9D for a detailed and incisive review. We agree that soundness is the binding issue, and addressed it directly rather than arguing that the method is better: the revision reports the complete costs we can measure, states where the method does not win, and narrows the claim to what the evidence supports.

W1 / Q1 — complete cost measurement, and whether the conclusion survives it. We accept that counting only training FLOPs cannot support a compute-efficiency claim. The previous accounting excluded the full-pool feedback pass, representative reassignment, weight recomputation, topology scoring, re-entry validation, and optimizer updates; all are now included for every method, so both arms are accounted identically. Net ΔFLOPs shift by at most 1.8 pp (HWFET MLP −48.8% vs −49.4% before), with the direction unchanged in all settings.

More importantly, we now report direct end-to-end wall-clock on a single RTX 4060 Laptop GPU, serialized, as the median over three paired seeds, covering clustering, the full-pool feedback pass, reassignment, weight recomputation, validation, checkpoint selection, and all training stages, for all 21 settings (Appendix H). For the US06 MLP setting, the measured medians are 482 s for Full, 667 s for Random, 692 s for GradMatch, and 642 s for Ours; for gas MLP 2014 the corresponding medians are 270, 14, 18, and 11 s, respectively. Across 21 settings the proposed arm is faster than full-data retraining in 19 (median ratio 0.04–0.82) and shorter than GradMatch in all 21 (0.05–0.93); it is within ±7% of Random in 20. The two settings slower than full-data are battery MLP LA92 and US06 — exactly those with the smallest FLOPs savings (−11.2% and −7.2%), where CPU-side overhead dominates. All timings were checked against the accounting; two logs with machine-contention spikes were re-run. Energy and cloud cost are not reported because power draw was not instrumented; GPU-seconds are the direct evidence and are not converted into a speculative cost figure.

We agree that reduced counted FLOPs can coexist with longer runtime, and revised the framing accordingly: the contribution is now presented as a task- and architecture-dependent accuracy–compute trade-off, not a compute-efficiency claim, with wall-clock reported so this can be checked directly. Under complete accounting the compute-saving conclusion holds in 19 of 21 settings, but is explicitly not a uniform speedup.

W2 / Q2 — breadth of the baseline study. The retention-matched comparison is no longer restricted to battery MLP: it now spans 3 architectures × 4 battery cycles + 3 gas-turbine years = 21 cells, three paired seeds each. On battery (12 cells) the proposed method beats Random in 8 and loses in 4; all four losses are within seed-level noise (|ΔMAE| ≤ 7%), while several wins are clearly separable (MLP/US06 −11.8%, CNN/HWFET −10.8%). On gas turbine (9 cells) we no longer claim an accuracy win: for MLP the method saves ≈76–85% of accounted FLOPs at ≤6.7% MAE cost, but at ~40% retention the accuracy cost is larger for FCN/CNN (up to ~13%), consistent with the higher sample demands of convolutional feature learning. Both favorable and unfavorable cases are now stated in the abstract-level claims and in the revised Conclusion; we prefer to report this honestly rather than claim generalization the data does not show.

On CCS, CRAIG, BWS and SELCON: we have not added them as baselines, and prefer to explain that choice rather than report a rushed adaptation. They target different objectives — CCS combines coverage with importance pruning; CRAIG/GLISTER approximate the full-data gradient or validation objective; BWS selects a contiguous window from difficulty-ranked samples; SELCON constrains validation error for L2-regularized regression. None formulates condition-specific, mass-preserving support over windowed regression inputs. A faithful comparison would require re-implementing each criterion for our condition-partitioned setting and re-tuning it under the same retention budget — changing both the selection criterion and the training protocol, so a one-off adaptation is not like-for-like evidence. We instead compare the families these methods represent on equal footing under the complete FLOP accounting: GradMatch for gradient-based selection (the family containing CRAIG/GLISTER), k-center and uniform random for geometry/coverage-based selection (the family containing CCS), and residual-guided vs. random re-entry under matched budgets for difficulty-based selection (the family containing BWS). The claim is narrowed to the condition-partitioned composition, explicit mass accounting, and residual feedback as a pipeline — not to a new coreset criterion that beats every named method. This scope is now stated in Appendix A and in the revised Conclusion.

All wall-clock measurements and the extended comparison are in the revised manuscript. We thank the reviewer again; these concerns improved the paper.

Thank you for the added cost measurements and the additional baseline study. They address my concerns. I will not change my score but increase my confidence.

Reviewer_4cmvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes condition-specific training-data sparsification for industrial regression. It selects geometric representatives, weights them by assignment counts, and uses residual feedback to reintroduce samples. Experiments on battery state-of-charge estimation and gas-turbine NOx prediction across MLP, FCN, and CNN architectures evaluate the trade-off between test MAE and analytical FLOPs at approximately 40% final retention.

Critical strengths

The held-out driving-cycle evaluation and the per-seed results in Appendix D make the accuracy–compute trade-off inspectable. The retention-matched comparisons in Table 2 also show where the method helps: it achieves lower mean MAE than both Random and GradMatch on LA92 and US06, while both controls achieve lower mean MAE on UDDS.

Critical weaknesses
  1. The FLOP accounting explicitly excludes full-pool feedback inference, representative reassignment, and weight recomputation. These costs matter to the end-to-end retraining claim, particularly for US06, where Table 2 reports a mean accounted reduction of 8.3% with a standard deviation of 16.2 percentage points.

  2. Table 3 isolates frequency weighting, but its re-entry comparisons also vary optimizer-state handling and checkpoint selection. The reported results therefore leave the incremental value of residual-guided re-entry uncertain.

Questions for authors
  1. What are the end-to-end retraining times on the same hardware for the proposed method, full-data training, and the retention-matched baselines, including feedback inference, reassignment, and weight recomputation? The US06 MLP setting would be particularly informative.

  2. With the initial weighted support, final retention budget, optimizer-state handling, and checkpoint rule fixed, how does residual-guided re-entry compare with random re-entry across paired seeds?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 4cmv

We thank Reviewer 4cmv for a careful review. The revision (i) completes the FLOP accounting symmetrically for all methods, (ii) re-runs all experiments end-to-end under the corrected scope, and (iii) adds the two experiments requested here: direct wall-clock timing on all 21 settings and a controlled residual-vs-random re-entry ablation on paired seeds.

W1 — FLOP accounting and the US06 dispersion. We accept this criticism fully; it was the most consequential issue in the paper. The previous accounting excluded five pipeline operations for the proposed method — full-pool feedback inference, representative reassignment via cKDTree, post-re-entry weight recomputation, topology/landmark scoring, and re-entry decision validation — and also excluded optimizer updates, common to all methods. All are now included, and both arms are accounted identically. Net ΔFLOPs move by at most 1.8 pp (e.g., HWFET MLP −48.8% now vs −49.4% before); the compute-saving conclusion is unchanged in all eight battery settings.

On US06 specifically: the large spread (±16.2 pp) came from inter-seed variation in baseline training length — the full-data model stops at 27, 45, and 27 epochs on seeds 42/2026/3026, so seed 2026 costs roughly 38% more baseline FLOPs. We now report results per seed instead of only as a mean. Under the corrected symmetric accounting, US06 MLP shows −7.2% mean total FLOPs, with the per-seed breakdown (+2.2%, −26.1%, +2.2%) given in the revised appendix. The effect is therefore no longer a mean masking a 16-pp spread; the spread is explained, quantified, and reported transparently.

Q1 — End-to-end retraining time (including feedback inference, reassignment, weight recomputation). We measured end-to-end retraining wall-clock on a single RTX 4060 Laptop GPU, serialized (no concurrent jobs), with three seeds and the median reported. The scope includes clustering, the full-pool feedback pass, reassignment, weight recomputation, validation, checkpoint selection, and all training stages. This covers all 21 main settings (12 battery: four cycles × three architectures; 9 gas-turbine: three years × three architectures); only component ablations are excluded. Complete four-arm tables are in Appendix H, Tables H.1–H.4.

For the requested US06 MLP setting, the four-arm medians are Full 482 s, Random 667 s, GradMatch 692 s, Ours 642 s (GPU-seconds, over seeds 42/2026/3026). Here the proposed arm is faster than both retention-matched baselines but slower than full-data retraining — this is one of only two settings where the relative FLOPs saving is smallest (US06 −7.2%, LA92 −11.2%), so the CPU-side selection overhead is proportionally largest and outweighs the reduced training FLOPs. Across all 21 settings, the proposed arm is faster than full-data retraining in 19 of 21 (median GPU-seconds ratio 0.04–0.82) and shorter than GradMatch in all 21 (ratio 0.05–0.93; GradMatch's refresh stage is especially costly on battery FCN/CNN). It is within ±7% of Random in 20 of 21 settings. All timings were checked against the FLOPs accounting (large FLOPs savings → large wall-clock savings; smallest savings on battery MLP LA92/US06 → near parity), and two logs with machine-contention spikes were re-run before reporting. Energy is not reported because power draw was not instrumented. We agree that counted FLOPs alone do not establish practical acceleration — precisely why the full timing tables are now included; the revised text frames this as an accuracy–compute trade-off, not a uniform speedup.

Q2 — Residual-guided vs random re-entry under matched conditions. We held fixed the initial weighted support, the final retention budget (γ = 40.4/39.2/39.7/41.4% for HWFET/LA92/UDDS/US06), optimizer-state handling, the checkpoint rule, and topology processing. The only variable is how re-entry candidates are ranked (residual-guided vs uniform random), with the number of re-entered samples identical by construction and verified per cycle (HWFET 3263, LA92 1352, UDDS 1987, US06 5618). Over 12 paired seed runs, residual guidance lowers test MAE in 8 of 12: random re-entry degrades MAE by +4.7% on average (median ≈+5%). The effect is largest and most consistent on UDDS (mean +12.0%; two seeds +18.0%/+18.7%) and LA92 (+7.2%; all three seeds positive); on HWFET the two are within seed noise (−2.3%). We also corrected the confound the reviewer identified: the non-re-entry vs re-entry rows in the original Table 3 changed retention and topology handling alongside feedback, and are now retention-, optimizer-, and topology-matched, so the incremental value of residual-guided re-entry is isolated.

All numbers and per-seed tables are in the revised manuscript and supplement (Appendix H for timing; the re-entry ablation table and Appendix D for paired seeds). We thank the reviewer again — both criticisms improved the paper.

We thank the authors for the revised accounting and the new experiments.

The symmetric FLOP accounting and the wall-clock measurements address our main concern. The timing also shows the size of the advantage: the method is faster than full-data retraining in 19 of 21 settings and faster than GradMatch in all 21, but within 7% of Random in 20 of 21. On US06 MLP it is slower than full-data retraining, and the mean FLOP saving there comes from one seed (+2.2%, −26.1%, +2.2%), so it depends on when the full-data baseline stops. Please state this in the paper.

The matched re-entry ablation isolates residual guidance as requested. It lowers MAE in 8 of 12 paired runs, mainly on UDDS and LA92, and is within noise on HWFET.

Our concerns are largely addressed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.