- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper introduces SparseEIS-Bench, a benchmark for reconstructing a full electrochemical impedance spectrum from three or five measured complex points, and proposes one method, MOH-FM, as a reference generative baseline. The data resource is large and by the paper's account newly measured: 25,090 spectra from 406 batteries, 1,218,599 raw frequency-point records across 21 capacity-chemistry folders as the paper counts them (23 cohort rows in the released index). The task is fixed in advance: a 512-point logarithmic frequency grid, a shape-preserving cubic (PCHIP) reference target, per-spectrum validity and observation masks, battery-disjoint splits, and 3/5-point observation budgets shared by all methods. Eight methods are compared on a pooled complex RMSE; MOH-FM reaches 0.0465 and 0.0359 Ohm at three and five points and is the lowest mean in both settings. MOH-FM adds to a deterministic predictor a residual flow whose path is reparameterised per Fourier mode by a coverage-dependent exponent, with measured values projected back at observed positions. Beyond accuracy, the paper reports Kramers-Kronig (KK) pass rates, equivalent-circuit parameter agreement, prediction-interval coverage, and a downstream state-estimation figure.
Critical strengths
- The data accounting and the protocol are verifiable to the digit, which is what makes a benchmark reusable.
From the cohort index shipped with the submission I can reproduce, exactly, both headline numbers of the data resource: the per-cohort spectrum counts sum to 25,090 and the per-cohort raw point counts sum to 1,218,599. The chemistry totals in Table 5 also reproduce from the cohort table once the three mixed cohorts are counted as the paper counts them: their 62, 76 and 52 spectra sum to the 190 "NCA/NCM mixed" spectra, and those are exactly the three cohorts that the split script handles jointly. The task protocol is fixed before any method sees the data, the observation masks are nested by construction, the split and mask generators are shipped, and the evaluator reports both pooled point errors and per-spectrum averages. For a benchmark paper this is the property that decides whether other groups can use it, and it is done better here than in most submissions in this area. It is also why my concerns below are about interpretation rather than about whether the data are real.
- The paper reports, and does not soften, the result that most deserves to survive revision.
The three-point checkpoint achieves the lowest aggregate error and yet fails a KK consistency check on 52.71% of spectra. The paper states this plainly ("nearly half the spectra from the three-point checkpoint fail this criterion", Section 5.2) instead of burying it, and it explicitly declines to claim passivity from finite-band KK testing. This is exactly the kind of finding a benchmark exists to produce, and the paper's own numbers make it stronger than the paper claims: the reference representation passes at a median residual of 0.001939 (Appendix D), so the gate at 0.05 admits spectra roughly 26x worse than the reference. I would rather see this result strengthened (per-seed) than removed.
Critical weaknesses
- It is unclear to me what the reported numbers measure.
The target is defined twice, and differently. Section 3.1 and Appendix A.4 define it as PCHIP interpolation of each measured spectrum onto a 512-point logarithmic grid, evaluated only inside a validity mask whose gaps stay below 0.35 decades, and Section 5.1 scores over grid positions with Equation (13). The supplementary protocol defines the same task as "the complete measured impedance response at that spectrum's original frequency points", states that "no interpolation, resampling, smoothing, or normalization is applied", and asks methods to "score predictions at the original target frequencies". In the first, most scored positions are spline output; in the second, every scored position is a measurement. No resampling code is in the release, so the artifact cannot reproduce Table 2 under the protocol the paper describes, and the paper never says which definition produced Tables 2 and 3. That question decides whether the task is "recover the measurement" or "fit a spline", which have different difficulty and different error floors.
The aggregate is dominated by the high-impedance end of the corpus. Equation 13 pools squared errors in physical units, so a spectrum's weight in the sum scales with the square of its impedance magnitude. The pool is already unbalanced in count, since the largest cohort is a 0.045 Ah LCO set holding 34.2% of all spectra, and it is further unbalanced in magnitude: in the two spectra I sampled, that cell has |Z| of about 0.39 Ohm at its highest measured frequency while a 628 Ah LFP cell has |Z| of about 1.3e-4 Ohm, three orders of magnitude apart and seven in squared error. On that ratio the reported 0.0465 Ohm exceeds the whole impedance of the large-format cell by more than two orders of magnitude, so the aggregate is in practice a statement about the small, high-impedance cells that dominate the pool. The capacity span the paper advertises (Table 1 and Table 4 report 0.04-628 Ah) therefore cannot be read into the headline number. The evaluator shipped with the submission already groups by cohort and already reports both pooled and per-spectrum averages; neither appears in the paper.
There is no error floor of either kind. No constant-predictor floor and no leave-one-measurement-out interpolation error are reported, so an aggregate in Ohm cannot be read as skilful or unskilful, and the spline reference's own error is unknown. The reference grid carries about 73 points per decade over the modelled band against roughly 10-12 measured points per decade, so on the order of six of every seven scored positions hold no measurement. The release's own protocol asks for per-cohort and per-chemistry scores, eligibility counts and battery-group resampling intervals; none of these appear in the paper.
I would revise this concern if the authors reported a per-cohort table with both pooled and per-spectrum averages, a relative error whose denominator is each spectrum's own magnitude, and a constant-predictor floor.
- The comparison does not support the claimed ranking.
Three of the four advertised axes are measured on one method. Every entry of Table 3, including the KK pass rate, equivalent-circuit agreement, interval coverage, NLL and CRPS, and the whole of Figure 6, are computed from "raw outputs from one checkpoint per budget", that is, from MOH-FM alone, and no baseline is scored on any of them. With a single method on each of these axes, the paper cannot show that they discriminate between methods.
The only baseline drawn from the literature is run outside the assumption that defines it, and what remains of the margin is inside the noise. Mänken et al. (2025) reconstruct a spectrum as U_r (C U_r)^-1 y, where the measurement matrix places its unit entries at r/2 frequencies read off the basis by a pivoted QR factorization whose stated purpose is "to ensure a stable inversion", and they note that adjusting those frequencies by hand "is incongruous with the original optimality, as evidenced by a divergent reconstruction quality". This submission fixes the observation locations to a logarithmic-equidistant set for every method (Equation 15) while Section 3.1 asserts that deterministic baselines "use the same sparse impedance inputs according to their definitions" - a sentence the baseline's own definition contradicts.
The same argument applies to a method that is absent entirely: Sun et al. (2023) predict the full spectrum from four DRT-derived characteristic frequencies, which they note must be re-derived when cell type or temperature changes; the submission does not cite it, and Section 2 has no paragraph on sparse-to-full reconstruction although that is the paper's own task.
The consequence is visible in the table. That baseline scores worst of eight (0.1225 and 0.0502, below both parameter-free interpolators at both budgets, and below the paper's own Direct DiT at three points), and it is named "Eigenspectra ridge" although Mänken et al. use no ridge. The paper also does not state which corpus the eigenspectra basis was trained on, although that method is scoped to cells "with similar material combinations". Even reading Table 2 at face value, the margin that carries the headline is thin: at five points the proposed method leads a parameter-free PCHIP interpolator through the same five points by 0.0013, about half a seed standard deviation, with no test reported, and at five points the method's own global-only ablation (0.0395) is worse than that interpolator; at three points its nearest competitor is again its own ablation, because four of the eight rows are variants of the proposed family.
I would revise this concern if the reliability axes were reported for at least two baselines, the eigenspectra baseline were reported at its own operating point alongside the fixed-location result, and a paired test on the per-cell difference were given.
References for the two works cited above. [1] Mänken, Schäfer and Eichel, "The exploitation of eigenspectra in electrochemical impedance spectroscopy: Reconstruction of spectra from sparse measurements", Journal of Power Sources, 2025. This one is in the submission's reference list, and it is the source of the baseline named "Eigenspectra ridge". [2] Sun, Xiong, Wang, Tian and Li, "Deep neural network based battery impedance spectrum prediction using only impedance at characteristic frequencies", Journal of Power Sources, 2023. Not cited by the submission.
Questions for authors
Q1 raises a disclosure and provenance issue in the supplementary material, which I flag for the AC as well as for the authors. Questions 2 to 4 are the ones that most affect my assessment; the rest are secondary.
Q1 (disclosure in the supplementary material, and the provenance it implies). The file docs/DATASET.md states that "the three related Tongji capacity/cathode cohorts share a split namespace so their batteries cannot cross partitions through cohort renaming". This names an institution inside a double-blind submission, and it contradicts two statements made elsewhere in the same package: the dataset index is described as containing "no source register", and the README states that "Source collection labels and private source notes are kept outside this package". Three things follow. The reference should be removed from the release. The authors should state whether that institution is their own affiliation or an external source for those three cohorts, which are the 190 spectra of the mixed cohorts in Table 5 (0.76% of the corpus). And if those spectra were obtained from an external source, that bears on contribution 1's statement that the 25,090 spectra were "measured by the authors", and on release terms, since Appendix A.6 promises all 25,090 processed spectra on acceptance while the release states it "makes no broader redistribution claim".
Q2 (target definition). Tables 2 and 3: were they computed on the 512-point PCHIP grid with the validity/registration masks, or at each spectrum's original measured frequencies? If the former, why does the supplementary protocol state that the target is the original measured frequency points and that no interpolation or resampling is applied? If the latter, how do Equations (13) and (15)-(16) enter the evaluation?
Q3 (the Eigenspectra baseline). (i) On which corpus was the eigenspectra basis trained - the original solid-oxide data, or the lithium-ion training split? (ii) What is the ridge term in the name, given that Mänken et al. invert (C U_r) directly, and was it introduced because the equidistant observation set makes C U_r ill-conditioned? (iii) Can you report the baseline at its own operating point, i.e. letting it select its own r/2 frequencies, alongside the fixed-location result? The difference between the two is the effect of the protocol rather than of the method.
Q4 (observation locations). Why are observation locations fixed to a logarithmic-equidistant set for all methods rather than taken from each method's own selection rule, when the two closest published methods both define their input by selecting frequencies? A concurrent benchmark on public data (see the reference at the end of this section) reports that the selection scheme, not the point count, is the dominant driver of error; if that holds here, the fixed scheme is what the benchmark is measuring.
Q5 (hyper-parameters). Which values of (lambda, p, lambda_local, p_local, gamma) produced Table 2? The paper states (1, 2, 0.1, 1, 1), while the released model core defaults to lambda_max = 4.0, local lambda_max = 0.5 and local power = 2.0. The "matched scalar power" row depends on this, since with lambda = 1 every per-mode exponent lies in [1, 2] and a matched scalar path would be hard to distinguish from the per-mode path, whereas with lambda = 4 it would not.
Q6 (reporting). Can you report, per cohort, both the pooled and the per-spectrum average complex RMSE (the shipped evaluator already computes both), together with the number of batteries in each split and the cohort-by-split table? In particular, what fraction of the reported 0.0465 Ohm is contributed by the three largest-capacity cohorts? Relatedly, can a single BatteryID appear under two cohort labels? Appendix A.1 describes Battery ID as the sole grouping key, whereas the released split script keys on cohort and battery jointly, which would allow one battery to land in two partitions.
Q7 (KK). The gate is 0.05 while the reference representation passes at a median residual of about 0.0019. What are the pass rates at 1% and 2%, and what is the spread of the 52.71% figure across the three seeds? Separately, what fraction of the 25,090 measured spectra fails the same test, and was any validity screening applied to the training data?
Q8 (downstream). Figure 6 compares reconstruction-based estimation against full-spectrum estimation. Can you add the control that matters for the claim, namely a head regressed directly on the three or five measured points with no reconstruction step? Without it, the downstream result cannot distinguish "reconstruction preserves the task" from "the measured points already carried the task".
Minor points. Table 4 reports 21 capacity-chemistry folders while the released index lists 23 cohorts; the abstract says "complete impedance spectrum" while the body restricts reconstruction to the measured band; Table 1 and Table 6 report the same coverage numbers; Section 5's "eight baselines" includes the proposed method. None of these affects my assessment on its own.
Reference for Question 4. Barnwal et al., "Benchmarking Battery EIS Reconstruction from Reduced Measurements: Accuracy, Physical Validity, and Task Preservation", SSRN preprint, 2026, not peer reviewed. It is concurrent work that the submission does not cite; I use it only for its comparison of measurement schemes, and it is independent of the two weaknesses above.