- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 3
Read review
Paper summary
The paper presents a benchmark of real operando iSCAT microscopy movies that capture the same battery electrode particles under various operating conditions. It includes 22 graphite particles tested at different charging rates and temperatures. The objective is to predict a particle's future image after a specific amount of additional charge has passed, using its last four frames and the known current as inputs. Evaluation is conducted through transfer splits based on rate, temperature, particle type, and cycle. The key contributions of this work are a comprehensive dataset; a controlled comparison of models that predict future behavior based on charge increments versus time increments; strong non-neural baselines such as the replay of the same particle’s previous behavior and a mean-only predictor; and an assessment of the PDE foundation models Poseidon and Walrus. The findings reveal that predicting charge-transfer performance more effectively across slow-to-fast charging is limited to a single training rate. Additionally, a direct-prediction U-Net consistently outperforms the best baselines on graphite, although results are less conclusive for NMC. Furthermore, forecasts struggle to accurately predict both the timing and location of particle transformations, which may be influenced by variability in repeated experiments.
Critical strengths
- The benchmark proposed in the paper is based on approximately 300 hours of operando iSCAT recordings, which capture the same particles under various charging rates, temperatures, and cycles. This allows for a clear distinction between changes in operating conditions and changes in the observed objects.
Critical weaknesses
- The paper is very difficult to follow due to its reliance on 100+ pages of appendices and 120+ tables, with many key claims supported only by appendix results. Findings labeled as "descriptive," "retrospective," or "exploratory" are often reported briefly, leading to confusion about which conclusions are well-supported.
- The proposed benchmark is framed as a test of whether learned particle dynamics transfer across operating conditions, but it doesn’t clearly explain why forecasting iSCAT images is the right way to measure this, or what a well-transferring forecaster would enable. The forecaster requires operando iSCAT frames as input, which means it can only be used in specialized laboratory settings and cannot be applied to deployed batteries. Its value therefore lies in research applications, such as replacing costly experiments at new conditions or guiding experimental design. Unfortunately, the paper doesn’t specify which of these it targets, nor does it justify the main task choices (4 history frames, forecast horizons, and raw optical contrast as the target) in relation to any of them.
Questions for authors
- Could the authors clearly indicate in the main text which claims are supported by resolved, prospective evidence and which are exploratory or retrospective? Could they also move the key supporting results from the appendix into the main text?
- The coupled step-and-history analysis in Appendix J found that fewer history frames gave higher skill than the default (e.g., with ). Could the authors report an ablation that varies k at a fixed step size, preferably on the transfer splits, to justify ?
- About the forecast horizons: What scientific or practical consideration motivated the horizons of 0.01 to 0.32 of the reference half-cycle capacity?
- Since the optical contrast is not a calibrated lithium fraction, why is the main focus on predicting raw images instead of quantities that have more physical relevance, such as transition timing or switch maps? Would the rankings of the models change if transition timing were the primary metric?
8 comments
Thank you for the response and the revision. Adding Table 3, which provides the evidence status for each finding, along with the fixed-step history ablation and clarification of the target use, addresses several of my concerns. I have a few remaining points I would appreciate clarification on:
-
Physical readouts vs. pixel skill: In the fixed-start comparison, which does not use the observed future, cross-rate replay matches the direct U-Net on transition timing (0.038 vs. 0.037) and produces better switch maps than both U-Nets. How do the authors reconcile this with Finding 2? Does it suggest that, for the stated purpose of guiding experiment selection, the learned models add little beyond simple replay?
-
History length: The new ablation varies only for the step-by-step U-Net, and on the headline C/20→C/2 split, one or two frames outperform four at Δ = 0.16. Does the direct U-Net’s lead over the best baselines hold for other history lengths?
-
Target use vs. task design: The stated use is forecasting from the first images at a new condition, but forecast origins start at = 0.14. Could the authors explain how the current task setup reflects that use, or whether forecasts from earlier origins were evaluated?
Reply to Reviewer iULc (follow-up)
Thank you very much for the follow-up. Here are the further clarifications you requested:
Readouts and Finding 2. Finding 2 is about pixel skill at leads up to 0.32 Qref, the range the direct U-Net was trained on. In the fixed-start comparison (origin at the frame at or before u = 0.14), 80–88% of the C/20 → C/2 transition targets lie beyond that range (Appendix H.2), and in this comparison replay is as good on timing and better on switch maps. In a retrospective check that starts each forecast just before its transition, within the trained range, the direct U-Net has the smaller timing error (0.039 vs 0.051; difference −0.012 [−0.024, −0.000]) and a higher but unresolved map correlation (0.18 vs 0.08; Tables 54 and 55). The two comparisons also differ in origins and targets, so they do not isolate the effect of lead length. On your second question: yes, for choosing experiments from the first images, the learned models we tested add little over replay from a fixed early start, and none reaches repeat agreement on these C/20 → C/2 lithiation targets. This is part of Finding 3, and it is why we recommend comparing learned forecasters with same-particle replay (Sec. 7).
History length. We trained the direct U-Net with 1, 2 and 8 frames (fixed step 0.005 Qref, seeds 0–2) on the four graphite splits of Table 60, and scored all history lengths and the baselines on the same windows. Lead over the strongest baseline at Δ = 0.16:
| Frames | C/20 → C/2 | C/2 → C/20 | New particles (fold 0) | Later cycles |
|---|---|---|---|---|
| 1 | +0.066 [0.016, 0.085] | +0.173 [0.143, 0.250] | +0.064 [0.048, 0.213] | +0.155 [0.106, 0.223] |
| 2 | +0.109 [0.033, 0.134] | +0.173 [0.143, 0.250] | +0.080 [0.065, 0.219] | +0.166 [0.110, 0.239] |
| 4 | +0.107 [0.034, 0.131] | +0.174 [0.145, 0.257] | +0.099 [0.085, 0.226] | +0.165 [0.117, 0.231] |
| 8 | +0.099 [0.035, 0.126] | +0.174 [0.145, 0.254] | +0.113 [0.089, 0.217] | +0.162 [0.109, 0.232] |
The lead holds for every history length (fold 0 has five particles and is descriptive). Unlike the step-by-step U-Net, the direct U-Net gains nothing from fewer frames: on C/20 → C/2, two and eight frames match four, and one frame is slightly lower (−0.040 [−0.048, −0.017]; one of its three seeds stopped early). At Δ = 0.02, four frames are best or equal-best on every split, and one frame falls below the best baseline on C/20 → C/2 (−0.078 [−0.098, −0.050]). With all five seeds on its own windows, the four-frame model reproduces the published C/20 → C/2 gain (+0.111 [0.046, 0.133]). The table, with the Δ = 0.02 results and the differences from four frames, is now Table 62 (Appendix I.5, p. 69) of the revised PDF. The code and job list are in the anonymous code repository (scripts/rebuttal/direct_history.py).
Earlier origins. Yes. Table 33 (p. 44) scores the trained models from requested origins starting at u₀ = 0.05. On C/20 → C/2, for origins between 0.05 and 0.14 the direct U-Net leads the best baseline by +0.21 [0.19, 0.25] at Δ = 0.02 and +0.17 [0.15, 0.20] at Δ = 0.16, more than at later origins, and the charge clock still leads the time clock. This uses a shifted origin lattice, so it is descriptive. The fixed-start transition forecasts above are the closest match to the use we describe and the lattice did not start earlier than u = 0.05.
Reply to Reviewer iULc
Thank you for saying clearly which parts were hard to follow; we organised the revision around them.
W1 and Q1. Evidence status in the main text, and key results moved there. Sec. 7 now opens with Table 3 (p. 10), which gives each finding with its key numbers, evidence status (defined in Sec. 4 and the caption) and location. Finding 1, the role of a second training rate and the graphite part of Finding 2 are resolved and prospective: a forecast uses only information available when it starts, within an offline benchmark whose outlines, alignment and part of the calibration are built from completed acquisitions (Appendix B.3). Some splits train on, or replay from, acquisitions recorded after the test acquisition; Appendix B.4 rescores replay with earlier recordings only, and Appendix B.5 records the origin of each reference capacity. The NMC part of Finding 2 is not resolved. Conversion order is resolved within one electrode, and its check against the acquisition history is resolved with all runs but not without the two recovery runs (Sec. 7). The transition forecasts are retrospective; full rollouts of the step-by-step U-Net from C/20 to C/2 also fall short of repeat agreement in lithiation (timing error 0.105 against 0.030), whereas in delithiation the two errors are similar (0.026 against 0.022; Table 51). The foundation-model comparison is exploratory. The main text also gains the cell-history checks (Sec. 7) and the acquisition timeline (Fig. 1e).
W2. What a transferring forecaster would enable, and the task choices. We target the second use you name, guiding experimental design, and Sec. 1 now says so. One slow half-cycle takes about a day to record (the movies here cover 544 hours, 527 of them new; Table 4), so forecasts from the first images at an untested rate or temperature could help decide which conditions to image. Their usefulness for choosing experiments has not been evaluated, and Finding 3 shows that slow-to-fast forecasts still miss when and where particles convert. Sec. 4 now gives the evidence behind each task choice (Q2–Q4 below).
Q2. History length at a fixed step. With the charge step fixed at 0.005 of the reference capacity, we varied k over 1, 2, 4 and 8 on four graphite splits, including both rate-transfer directions, and one NMC split (three seeds, paired windows; Appendix I.5, Tables 60–61). At Δ = 0.02, four frames are best or equal-best on every graphite split: one frame loses 0.11–0.25, and two or eight lose at most 0.03. At Δ = 0.16 the effect depends on the split: from C/20 to C/2 other lengths beat four by 0.04–0.08 (resolved for one and two frames), four frames are best on new-particle fold 0 (five particles, descriptive), and the other differences are about 0.02 or smaller. On NMC, eight frames beat four by 0.02 [0.01, 0.04] at Δ = 0.02. The +0.13 for k = 2 in the former Appendix J came from the same-rate C/20 split, where the step size changed too; at a fixed step, two frames gain at most 0.04 [0.04, 0.05] (C/20 → C/2, Δ = 0.16). Four frames are therefore a reasonable choice at short horizons on graphite.
Q3. What motivated the horizons. The horizons are a doubling grid from 0.01 to 0.32 of the reference capacity, a benchmark convention whose endpoints were not derived from the physics; Sec. 4 now simply states what they span. The shortest covers a few stored frames at every rate, and the longest about a third of a slow half-cycle and the whole of the shortest half-cycles (C/2 at 6 °C). Measured afterwards on the same-condition repeats, the strongest optical change takes a median of 0.05 to 0.11 of the reference capacity to sweep across a particle (Appendix C), so the two shortest horizons fall within one conversion and the two longest exceed it.
Q4. Why images, and would the rankings change under transition timing? We forecast images because they keep the pattern within each particle, and pixel error, transition timing and switch maps all follow from one image forecast, whereas a timing model gives no spatial information. The rankings change in part, as Sec. 6.3 now says. For C/20 → C/2 lithiation forecast from a fixed start, where 80–88% of the targets lie beyond the direct U-Net's trained horizons, the direct U-Net and cross-rate replay have similar timing errors (0.037 and 0.038), the step-by-step U-Net is worst (0.073), and replay has better switch maps (ρ 0.14 against 0.01 for both U-Nets), so timing as the primary metric would keep the direct U-Net ahead of the step-by-step U-Net and similar to replay. In a retrospective check that starts each forecast just before its transition, within the trained range, the direct U-Net has the smallest timing error (0.039 against 0.051 for replay; difference −0.012 [−0.024, −0.000]) and a higher but unresolved map correlation (0.18 against 0.08). No forecaster reaches repeat agreement (0.025 and 0.33; Tables 54 and 55).