acceptodds
Under review as a conference paper at ICLR 2027

DuoEvo: Recursive Self-Improvement via Heterogeneous Challenger-Solver Co-Evolution for Industrial Fault Diagnosis

Abstract

Industrial automation requires reliable fault diagnosis under changing sensing conditions. Yet developing robust diagnostic models still relies on expert choices of training disturbances and network architectures. Recursive self-improvement (RSI) offers a route to automating model development, but most recent approaches leave the underlying neural architecture fixed. We introduce DuoEvo, a heterogeneous Challenger–Solver co-evolution framework that integrates challenge discovery with joint adaptation of model parameters and source-level structure. A Large Language Model(LLM)-based agent proposes executable noise programs and architecture revisions. Guided by the deployed Solver's weaknesses, the Challenger evolves a training curriculum; the Solver updates its parameters and evaluates structural revisions under matched curricula and budgets. Validation gates control deployment, and a patience rule retires unproductive structural search while challenge evolution and parameter learning continue. Each deployed Solver then defines the next round's challenge fitness, closing the recursive loop. On three bearing-diagnosis datasets, DuoEvo achieves the highest mean balanced accuracy among the compared methods on held-out synthetic corruptions under equal deployment-training budgets, with search costs reported separately. Fixed-reference evaluation shows bidirectional progress across all nine trajectories. Controlled analyses attribute most gains to Challenger evolution and a smaller, less certain contribution to structural adaptation. These results support a supervised self-improvement process that automates challenge discovery and model revision within a single feedback loop.

Primary area:
learning on time series and dynamical systems
Keywords:
Recursive self-improvement, Challenger--Solver co-evolution, LLM-based agents, Industrial fault diagnosis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8g7mOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DuoEvo is an industrial fault diagnosis framework, built upon heterogeneous Challenger–Solver co-evolution. It introduces executable noise programs, Solver-conditioned curricula, validation-gated architectural revision, and patience-based search termination.

Critical strengths
  1. Reframing robust diagnosis as jointly adapting training challenges and model structure, rather than relying on parameter updates alone, is interesting.

  2. Using fixed-reference evaluation and curriculum replay makes the evaluation more informative than relying on contemporaneous opponent scores.

Critical weaknesses
  1. DuoEvo spends many additional training epochs searching for better models. Without giving the baselines a comparable total budget, it is unclear whether the gains come from the proposed method or the extra search.

  2. The paper does not compare against a strong noise curriculum generated once and then kept fixed. This leaves unclear whether updating the Challenger every round is actually necessary.

  3. After controlling for retraining, the confidence intervals for architecture-related gains include zero on all three datasets. The evidence therefore does not establish that changing the architecture consistently improves performance.

  4. The non-degradation guarantee requires checking each replacement model before deployment. This check is skipped when reusing the round-3 checkpoint and rebuilding after the switch audit, so the guarantee does not cover these cases.

  5. The main experiments use synthetic noise and test data from the same operating conditions as training. They do not establish whether the method generalizes to new loads, new speeds, or real industrial noise.

Questions for authors

Q1. Could you clarify how the Challenger-progress argument is justified when the fixed Solver panel has unequal training exposure to early and late challenges?

Q2. I’m curious whether online recursion gives anything beyond a strong frozen curriculum in this setting. It is presented as the central mechanism, but the reported ablation does not separate feedback benefits from program-pool quality.

2 comments

Thank you for your constructive feedback. We have added experiments and analyses to address your questions.

Results report balanced accuracy (BA, %) and differences in percentage points (pp). Unless stated otherwise, tables give mean ± sample SD across seeds 11, 23, and 37. New evaluations use HN and round-20 checkpoints; HN averages four held-out noise families at 0, −3, and −6 dB.

Q1: How is Challenger progress measured without unequal training exposure?

We evaluate early and late challenges using a fixed external Random-Noise Solver panel whose models have not trained on the corresponding online curriculum. In Appendix D.1, mean classification margins decrease on all three datasets, supporting increased difficulty of later noise programs against the same reference models.

DatasetEarlyLateSeeds with lower late margin
PU0.6820.6552/3
CWRU0.6180.5382/3
JNU0.8800.7723/3

Lower classification margin indicates harder challenges for the same reference classifier. Mean margins decrease on all three datasets, providing evidence of progression beyond the internally exposed panel.

W2 & Q2: What does online recursion provide beyond a strong frozen curriculum?

Full DuoEvo achieves higher HN mean BA than the strong MAP-Elites frozen curriculum on all three datasets, with larger gains under severe interference. Gains are 0.50, 0.75, and 1.42 pp on PU, CWRU, and JNU, increasing to 1.68, 1.74, and 2.09 pp at −6 dB.

A frozen program library provides diverse disturbances selected against the initial Solver. Parameter learning and architectural updates change the Solver, motivating renewed challenge discovery throughout training. DuoEvo uses feedback from the updated Solver to guide subsequent noise-program synthesis and selection, tailoring training disturbances to dataset-specific diagnostic difficulties and the model’s current weaknesses. This feedback connects challenge discovery with model development and supports robustness to held-out noise families.

We constructed a strong frozen-curriculum baseline by generating programs against a fixed initial Solver, retaining them through MAP-Elites, and freezing the resulting library throughout classifier training. Both methods use identical source-condition test inputs. Both methods use 648 deployment-training epochs; the main-table frozen curricula use continuous cosine, and full DuoEvo uses round-wise cosine restarts.

DatasetMethod0 dB−3 dB−6 dBHN mean
PUStrong frozen curriculum99.57 ± 0.0397.54 ± 0.1491.37 ± 0.6696.16 ± 0.26
PUDuoEvo99.36 ± 0.2197.56 ± 0.4493.05 ± 0.8196.66 ± 0.48
CWRUStrong frozen curriculum99.71 ± 0.1698.32 ± 0.7393.60 ± 1.7197.21 ± 0.85
CWRUDuoEvo99.66 ± 0.0898.87 ± 0.3195.34 ± 0.8097.96 ± 0.39
JNUStrong frozen curriculum98.63 ± 0.5695.69 ± 1.6186.70 ± 3.6493.67 ± 1.70
JNUDuoEvo99.27 ± 0.0597.22 ± 1.0488.79 ± 5.2895.09 ± 2.10

These results evaluate the complete DuoEvo framework against the frozen curriculum, capturing the combined effects of adaptive challenge generation, parameter learning, and conditional architectural adaptation.

Evaluation at −7/−8 dB using the same checkpoints without retraining yields gains of 2.25/3.25 pp on PU, 0.89/0.46 pp on CWRU, and 3.40/3.85 pp on JNU over the frozen curriculum.

W1: Could additional search compute explain the performance differences?

We report development-time search costs alongside the matched deployment-training budget. DuoEvo performs additional search during model development; the final classifier requires no additional search computation at inference.

Deployment training uses 648 epochs; total training OEE also counts screening, confirmation, promotion rebuilds, and switch audits as an aggregate training-epoch measure.

DatasetDeployment OEETotal training OEERecorded elapsed hoursLLM calls
PU6483,320–4,58424.83–36.34480–560
CWRU6482,544–4,1923.38–6.11440–520
JNU6482,544–4,7443.54–9.85440–560

Structural search accounts for 74.5–86.3% of total training OEE. Recorded elapsed time includes API waits and machine-load effects. The accuracy comparison matches the 648 deployment-training epochs, with total development cost reported above. ClassBD and applicable evolved models retain auxiliary losses; MART+LORE disables them.

W3: How should structural effects be interpreted when intervals include zero?

All seven complete evolutionary trajectories that underwent architecture replacement have higher final HN BA than Evolve-NoStruct; the five retaining replacement architectures average a gain of 1.03 pp. The table groups the original manuscript’s existing seeds by replacement events and describes complete paths, including curriculum evolution and subsequent training.

The table summarizes round-20 HN BA (%) from the original manuscript, Appendix Table 11, for trajectories that underwent architecture replacement; gains are DuoEvo − Evolve-NoStruct (pp).

Dataset / seedDuoEvoEvolve-NoStructGain (pp)
PU / 1196.5894.74+1.84
PU / 2397.1796.37+0.80
PU / 3796.2695.68+0.58
CWRU / 1198.2196.59+1.62
CWRU / 3797.9896.87+1.11
JNU / 2396.1695.32+0.84
JNU / 3796.0893.11+2.97

CWRU seed11 and JNU seed37 later returned to the initial architecture; their endpoints reflect the complete replacement-and-rollback paths.

LLM proposals, candidate training, and curriculum evolution make gain magnitudes trajectory-dependent; validation gates, switch audits, and patience rules control replacement and search termination.

Appendix E's Anchor-Retrain analysis further estimates architecture-path effects of +1.65/+0.10/+1.45 pp on PU/CWRU/JNU, with paired 95% intervals [−1.61, 4.92]/[−0.33, 0.53]/[−4.79, 7.69], including subsequent training interactions. All three intervals include zero, indicating uncertainty in the average architecture-path effect.

W4: Does the final-checkpoint condition cover confirmation reuse and switch audits?

We audited confirmation-weight reuse and switch-audit branches to clarify the conditions required for the deployment non-degradation proposition. The guarantee requires the final deployed checkpoint to pass the same-round gate comparison against the continued incumbent.

Dataset/seed/roundBranchGate-mean changeClean changeMinimum family change
PU/11/3Confirmation-weight reuse+1.5920.000−0.270
JNU/37/3Confirmation-weight reuse−0.2670.000−1.900
CWRU/11/5Restore anchor after switch audit−1.044−0.147−5.930
JNU/37/6Restore anchor after switch audit−0.3030.000−3.050

The last three original deployments fail the gate-mean non-decrease condition, so the monotonicity guarantee does not cover those paths; a uniform final check would retain the continued incumbent. We will make the final deployed-weight check an explicit precondition of the proposition. The reported trajectories have not been rerun with this check; the audit and original HN results describe the original executions.

W5: How does evaluation relate to new operating conditions and industrial noise?

Our evaluation focuses on robustness to unseen synthetic interference under the training operating conditions. Executable noise programs model interpretable industrial interference structures; cross-load or speed transfer also changes the fault-signal distribution.

The noise primitives are motivated by industrial interference structures: colored/band-limited components represent spectrally concentrated background vibration, harmonics represent periodic interference from nearby rotating machinery or electrical coupling, resonant impulses represent impact-induced structural ringing, and bursts/modulation represent intermittent disturbances and varying interference intensity. Held-out swept_chirp, mult_speckle, heavy_burst, and ood_mix test frequency drift, signal-dependent multiplicative disturbance, heavy-tailed bursts, and their composition through separate implementations. Calibrated SNRs and reproducible realizations enable controlled robustness comparisons across these disturbance structures.

Reviewer_CGvyOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a recursive self-improvement strategy called DuoEvo. They propose a method that jointly finds training corruptions and learning the parameters of a neural network that learns those corruptions. An LLM proposes noise programs and changes to the network architecture and a current neural network scores those noise programs. Selected programs generate a training curriculum, on which the classifier’s weights and candidate architectures are trained. Experiments on three fault diagnosis datasets show that DuoEvo improves fault classification under artificial noise.

The presentation is very weak, with confusing figures and insufficiently clear explanations of the main components

Critical strengths
  1. The framework connects the classifier’s current weaknesses to the training disturbances from the noise, while allowing changes to both its weights and architecture.
  2. The controlled experiments are performed carefully (except baseline comparison, see below).
Critical weaknesses
  1. The proposed contribution is to adapt training corruptions and the model architecture jointly. However, the experiments do not establish whether repeatedly generating noise against the updated classifier is better than generating a strong noise collection once and keeping it fixed.
  2. Also the rationale for proposing such noise programs is not well motivated.
  3. The marginal gains in improvements are negligible compared to substantial additional training for architecture search.
  4. Different learning rate schedules further complicate fair comparisons with baseline.
Questions for authors
  1. How does DuoEvo compare with generating a strong noise collection offline and then training on that fixed collection? Is the method beneficial even when learning rates are tuned properly?
  2. Performing stability checks across additional seeds can establish the reliability of the gains of architecture search.
3 comments

Thank you for your constructive feedback. We have added experiments and analyses to address your questions.

Results report balanced accuracy (BA, %) and differences in percentage points (pp). Unless stated otherwise, tables give mean ± sample SD across seeds 11, 23, and 37. New evaluations use HN and round-20 checkpoints; HN averages four held-out noise families at 0, −3, and −6 dB.

W1 & Q1: How does DuoEvo compare with a strong frozen curriculum?

Full DuoEvo achieves higher HN mean BA on all three datasets, improving by 0.50, 0.75, and 1.42 pp on PU, CWRU, and JNU; at −6 dB, the gains are 1.68, 1.74, and 2.09 pp. Both methods use identical source-condition test inputs and 648 deployment-training epochs.

DatasetMethod0 dB−3 dB−6 dBHN mean
PUStrong frozen curriculum99.57 ± 0.0397.54 ± 0.1491.37 ± 0.6696.16 ± 0.26
PUDuoEvo99.36 ± 0.2197.56 ± 0.4493.05 ± 0.8196.66 ± 0.48
CWRUStrong frozen curriculum99.71 ± 0.1698.32 ± 0.7393.60 ± 1.7197.21 ± 0.85
CWRUDuoEvo99.66 ± 0.0898.87 ± 0.3195.34 ± 0.8097.96 ± 0.39
JNUStrong frozen curriculum98.63 ± 0.5695.69 ± 1.6186.70 ± 3.6493.67 ± 1.70
JNUDuoEvo99.27 ± 0.0597.22 ± 1.0488.79 ± 5.2895.09 ± 2.10

We constructed a strong frozen-curriculum baseline by generating programs against a fixed initial Solver, retaining them through MAP-Elites, and freezing the resulting library throughout classifier training.

The frozen library selects programs against the initial Solver, whereas DuoEvo updates its challenge distribution using feedback from the evolving Solver, tailoring disturbances to dataset-specific diagnostic difficulties and current model weaknesses to improve held-out-noise generalization.

These results evaluate the combined effects of adaptive challenge generation, parameter learning, and conditional architectural adaptation.

Evaluation at −7/−8 dB using the same checkpoints without retraining yields gains of 2.25/3.25 pp on PU, 0.89/0.46 pp on CWRU, and 3.40/3.85 pp on JNU over the frozen curriculum.

W4: How does learning-rate choice affect the comparison?

We conducted a controlled learning-rate comparison and used independent validation to select schedules for frozen curricula and the online curriculum variant. Both restart and continuous cosine were evaluated with fixed ClassBD architectures. The main-table frozen curricula use continuous cosine on all three datasets; on CWRU/JNU, this schedule was selected by independent validation seed 97.

Full DuoEvo uses round-wise cosine restarts. The online arm in the added schedule experiment is a separate fixed-backbone variant: under the selected schedules, its HN BA is below the frozen curriculum by 0.49/0.58 pp on CWRU/JNU. This result evaluates the fixed-backbone variant, while the main table evaluates full DuoEvo with conditional architectural adaptation.

Each round starts a fresh cosine cycle to adapt to its updated curriculum while carrying model weights forward. The 648-epoch budget comprises 8 bootstrap epochs and twenty 32-epoch segments.

To examine learning-rate choice for frozen and online curricula, we compared restart and continuous cosine at peak learning rate 0.01 with fixed ClassBD architectures, 320 program proposals, common augmentation/validation rules, and momentum resets at segment boundaries. This comparison uses an online curriculum variant with revised coverage/progress selection and a fixed backbone. Independent validation seed 97 selects continuous cosine for frozen training on both datasets and for the online variant on JNU, and restart for the variant on CWRU. Reporting seed 11 gives online-variant/frozen HN BA of 96.52/97.01% on CWRU and 94.13/94.71% on JNU.

ClassBD and applicable evolved models retain auxiliary losses; MART+LORE disables them.

Q2: Do additional seeds support reliable architectural improvement?

Our per-seed analysis shows higher final HN BA than Evolve-NoStruct in all seven complete evolutionary trajectories that underwent architecture replacement. Among the nine existing runs, five retain replacement architectures at the endpoint; all five improve, averaging +1.03 pp.

This table further analyzes the original manuscript's existing seeds (11, 23, and 37 per dataset), rather than additional independent search runs. The results describe complete training trajectories grouped by replacement events, including curriculum evolution and subsequent training.

The table reports round-20 HN BA (%) from Appendix Table 11; gains are DuoEvo − Evolve-NoStruct (pp).

Dataset / seedDuoEvoEvolve-NoStructGain (pp)
PU / 1196.5894.74+1.84
PU / 2397.1796.37+0.80
PU / 3796.2695.68+0.58
CWRU / 1198.2196.59+1.62
CWRU / 3797.9896.87+1.11
JNU / 2396.1695.32+0.84
JNU / 3796.0893.11+2.97

CWRU seed11 and JNU seed37 later returned to the initial architecture; their endpoints reflect the complete replacement-and-rollback paths.

LLM proposals, candidate training, and curriculum evolution make gain magnitudes trajectory-dependent; validation gates, switch audits, and patience rules control replacement and search termination.

Appendix E's Anchor-Retrain analysis further estimates architecture-path effects of +1.65/+0.10/+1.45 pp on PU/CWRU/JNU, with paired 95% intervals [−1.61, 4.92]/[−0.33, 0.53]/[−4.79, 7.69], including subsequent training interactions. All three intervals include zero, indicating uncertainty in the average structural contribution.

W3: How should gains be interpreted relative to search cost?

DuoEvo performs search during model development, while the final deployed classifier requires no additional search computation at inference. We separately account for development costs, including structural search, candidate training and evaluation, and LLM calls.

Total training costs are 2,544–4,744 OEE, with structural search accounting for 74.5–86.3%; the comparison matches 648 deployment-training epochs.

OEE aggregates training epochs across deployment training, screening, confirmation, promotion rebuilds, and switch audits; it is not a hardware-normalized compute measure.

DatasetDeployment OEETotal training OEERecorded elapsed hoursLLM calls
PU6483,320–4,58424.83–36.34480–560
CWRU6482,544–4,1923.38–6.11440–520
JNU6482,544–4,7443.54–9.85440–560

Recorded elapsed time includes API waits and machine-load effects; LLM calls are reported separately.

W2 & presentation: Why these noise programs, and how do the components interact?

We use executable noise programs to represent interpretable interference patterns and adapt them through feedback from the current Solver. Each round freezes the Solver for program evaluation, then freezes the selected curriculum for parameter learning and optional architecture search; the selected Solver guides the next round.

The noise primitives are motivated by industrial interference structures: colored/band-limited components represent spectrally concentrated background vibration, harmonics represent periodic interference from nearby rotating machinery or electrical coupling, resonant impulses represent impact-induced structural ringing, and bursts/modulation represent intermittent disturbances and varying interference intensity. Held-out swept_chirp, mult_speckle, heavy_burst, and ood_mix test frequency drift, signal-dependent multiplicative disturbance, heavy-tailed bursts, and their composition through separate implementations. Calibrated SNRs and reproducible realizations enable controlled robustness comparisons across these disturbance structures.

Programs receive a random-number generator, sample rate, and length, without signal labels; outputs are scaled to fixed attack SNR. Each program generates multiple reproducible noise realizations.

The LLM proposes code with fixed weights; supervised learning updates the classifier. HN is report-only. We will revise the main diagram to show the execution order, feedback edge, and optional structural branch.

Reviewer_CGvyOpenReview ↗

Reviewer acknowledgement

Thanks for the authors reponse. The new frozen baseline makes the paper clearer and seems to be a real improvement. The full DuoEvo does outperform the baselines. However, the comparison still do not tell whether updating the noise in response to the classifier is what actually helped.

I think Full DuoEvo adapts the noise, arcthiecture and change the leanring rate restarts. But the frozen baseline uses a fixed architecture and a continous leanring rate schedule. Clean comparison would have been to keep the architecture search and traning schedule the same for both sides.

While I thank the authors for their detailed comparison, I would still stick to rejection.

Reviewer_Ekf7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DuoEvo, an automated development framework for noise-robust industrial fault diagnosis. A fixed LLM generates executable corruption programs and neural architecture revisions. The Challenger searches for corruptions that reduce the current Solver's classification margin, and the selected programs form a training curriculum. The Solver continues supervised parameter learning and conditionally adopts architectural changes after screening, confirmation, and validation gates. Each deployed Solver determines challenge fitness in the following round. A patience rule permanently terminates structural search while curriculum evolution and parameter learning continue.

Experiments on PU, CWRU, and JNU report held-out-noise balanced accuracies of 96.67%, 97.87%, and 94.91%, respectively, the highest aggregate means among the evaluated methods. Fixed-reference cross-generation evaluations measure reciprocal progress, while curriculum replay and event-matched anchor retraining investigate the sources of improvement. The authors conclude that most gains come from Challenger evolution, with a smaller and less certain contribution from structural adaptation.

Critical strengths
  1. A coherent application-driven design. Jointly adapting corruption coverage and model representation is well motivated for noisy vibration-based diagnosis. Fixed-SNR challenge evaluation prevents trivial fitness gains from increasing noise power alone. (§3.1)

  2. Careful attribution beyond ordinary component ablations. Evolve-NoStruct, byte-identical Replay-Anchor, and event-matched Anchor-Retrain address distinct questions about curriculum evolution, structural paths, and retraining. The exact-zero replay results for no-promotion runs provide a useful consistency check. (§4.4; Appendix E)

  3. Appropriate evaluation of co-evolutionary progress. The cross-generation matrices compare generations against common references rather than relying exclusively on moving-opponent scores. The additional Random-Noise Solver panel supplies a useful external reference. (§4.3; Appendix D.1)

  4. Promising results and transparent reporting. DuoEvo achieves the highest reported aggregate HN means on three datasets, with substantial improvements over static ClassBD at severe noise levels. The authors disclose total search costs, statistical uncertainty, same-condition evaluation, and preprocessing limitations. Held-out generators are excluded from online model selection. (Tables 1, 14, and 16)

Critical weaknesses

W1. The distinctive benefit of online recursive feedback is not isolated

Random-Noise removes LLM program variation, archive retention, and Solver-conditioned selection together; it also changes program coverage and reuse. It therefore identifies the contribution of the Challenger branch as a package, not the individual contribution of feedback or LLM generation. Fixed-reference cross-generation progress supports the observation that both sides improve, but does not establish that online feedback outperforms a sufficiently diverse curriculum evolved offline and then frozen. Replay-Anchor inherits the online donor's curricula and cannot answer this question. The authors acknowledge this limitation, but it remains central to evaluating the recursive mechanism. (§4.4; Appendices C.3 and D.1)

W2. The incremental structural contribution remains uncertain

DuoEvo exceeds Evolve-NoStruct by 1.07, 0.70, and 0.77 percentage points on PU, CWRU, and JNU, respectively; all three paired 95% intervals cross zero. (Table 14) The donor-conditioned replay effects are positive on average, but also include retraining effects. After the Anchor-Retrain decomposition, the architecture-path means and intervals are:

DatasetArchitecture-path mean gainPaired 95% interval
PU+1.65 pp[−1.61, 4.92]
CWRU+0.10 pp[−0.33, 0.53]
JNU+1.45 pp[−4.79, 7.69]

In CWRU seed 11 and JNU seed 37, the reported replay difference is entirely assigned to the retraining component. Moreover, the architecture-path estimate includes interactions with subsequent training under the donor curriculum, rather than being an architecture-only main effect. These findings do not demonstrate ineffectiveness, but three seeds provide limited evidence for a reliable incremental structural benefit. (Appendix E.1; Table 13)

W3. The performance return on structural-search cost is unclear

The comparison matches 648 deployment-training epochs, whereas total DuoEvo training costs are 2,544–4,744 OEE, approximately 3.93–7.32 times that epoch count, plus 440–560 LLM calls. OEE is training-epoch accounting, not a hardware-normalized FLOP or runtime measure. In a search round with three confirmed sources, screening and confirmation alone cost 40 × 8 + 3 × 104 = 632 OEE, before rebuilds and audits. Calculated from Table 16, structural training accounts for 74.5%–86.3% of total OEE. (Table 4; Appendix F)

DatasetTotal OEE per runRecorded elapsed timeLLM calls
PU3,320–4,58424.83–36.34 hours480–560
CWRU2,544–4,1923.38–6.11 hours440–520
JNU2,544–4,7443.54–9.85 hours440–560

These ranges summarize the three runs per dataset in Table 16. Recorded elapsed time depends on the machine and load; it is not a standardized comparison with baseline runtime. OEE also does not capture all challenge-evaluation and LLM costs.

The authors disclose the additional expense clearly. My concern is whether the uncertain gains over Evolve-NoStruct justify it. A comparison against fixed-architecture retraining, hyperparameter optimization, or augmentation search given equivalent total resources would clarify the practical value of structural search. An accuracy-versus-total-cost curve, with actual LLM usage reported separately, would be particularly useful.

The extra expense arises primarily during development: the final network does not require LLM calls for each prediction. Reuse could amortize it, but cross-condition reuse and human time savings are not quantified. Thus, automation does not by itself establish reduced overall development cost. Interpretation is further complicated by cosine restarts in adaptive training versus a single cosine schedule for static baselines, and by disabling ClassBD auxiliary losses in MART+LORE while retaining them in ordinary ClassBD and applicable evolved models. (Appendices C.2 and C.5)

W4. The deployment guarantee is conditional and does not cover every executed branch

Proposition 1 requires the final deployed checkpoint to match or exceed the continued incumbent's gate score. Appendix B.1 explicitly notes that round-3 reuse of confirmation weights and the switch audit do not fully verify this condition. The proof is valid under its premises, but those premises are not enforced throughout the executed policy. The authors should clarify the observed behavior of these branches and distinguish source-level acceptance from final-checkpoint non-degradation. The guarantee concerns the gate score, not held-out generalization. (Appendices A, B.1, and C.2)

W5. The evaluation leaves an important preprocessing question and a limited generalization scope

The reported HN results concern synthetic corruption transfer under the training operating conditions; the reserved cross-condition data are not evaluated. The authors correctly acknowledge this scope. Boundary purging also appropriately removes raw-sample overlap across temporal splits. However, CWRU and JNU normalization uses complete source records before splitting, including held-out temporal segments. This shares unlabeled statistics across splits, and its impact cannot be determined from the absence of overlapping windows. A training-only-normalization sensitivity check would strengthen confidence in the evaluation. Testing the already reserved operating conditions would additionally clarify transfer beyond same-condition corruption robustness. (Appendix C.1; Table 3)

Questions for authors
  1. Online feedback (W1): Does online Challenger adaptation outperform a curriculum evolved against a fixed initial Solver and subsequently frozen, under matched program-generation and training budgets?

  2. Structural attribution (W2): How stable is the architecture-path difference relative to Anchor-Retrain across additional independent search runs, and which architectural changes account for the improvements?

  3. Cost-effectiveness (W3): Under the same total compute budget, does DuoEvo outperform fixed-architecture retraining or augmentation/hyperparameter search? Please provide an accuracy-versus-total GPU time or FLOPs comparison, report LLM usage separately, and clarify the effects of learning-rate restarts and auxiliary-loss settings.

  4. Deployment checks (W4): After round-3 reuse and switch audits, what are the changes in final-checkpoint gate mean, clean accuracy, and per-family accuracy relative to the continued incumbent? Would applying the same final-checkpoint check to these branches alter the reported trajectories?

  5. Preprocessing and scope (W5): Do the main comparisons persist when normalization uses training-only statistics? If available, how do the models perform on the cross-condition splits already reserved in Table 3?

2 comments

Thank you for your constructive feedback. We have added experiments and analyses to address your questions.

Results report balanced accuracy (BA, %) and differences in percentage points (pp). Unless stated otherwise, tables give mean ± sample SD across seeds 11, 23, and 37. New evaluations use HN and round-20 checkpoints; HN averages four held-out noise families at 0, −3, and −6 dB.

W1 & Q1: Does online adaptation improve over a strong frozen curriculum?

Yes. Full DuoEvo outperforms the strong MAP-Elites frozen curriculum on PU, CWRU, and JNU, with HN mean BA gains of 0.50, 0.75, and 1.42 pp, increasing to 1.68, 1.74, and 2.09 pp at −6 dB.

A frozen program library provides diverse disturbances selected against the initial Solver. Parameter learning and architectural updates change the Solver, motivating renewed challenge discovery throughout training. DuoEvo uses feedback from the updated Solver to guide subsequent noise-program synthesis and selection, tailoring training disturbances to dataset-specific diagnostic difficulties and the model’s current weaknesses. This feedback connects challenge discovery with model development and supports robustness to held-out noise families.

We constructed a strong frozen-curriculum baseline by generating programs against a fixed initial Solver, retaining them through MAP-Elites, and freezing the resulting library throughout classifier training. Both methods use identical source-condition test inputs.

DatasetMethod0 dB−3 dB−6 dBHN mean
PUStrong frozen curriculum99.57 ± 0.0397.54 ± 0.1491.37 ± 0.6696.16 ± 0.26
PUDuoEvo99.36 ± 0.2197.56 ± 0.4493.05 ± 0.8196.66 ± 0.48
CWRUStrong frozen curriculum99.71 ± 0.1698.32 ± 0.7393.60 ± 1.7197.21 ± 0.85
CWRUDuoEvo99.66 ± 0.0898.87 ± 0.3195.34 ± 0.8097.96 ± 0.39
JNUStrong frozen curriculum98.63 ± 0.5695.69 ± 1.6186.70 ± 3.6493.67 ± 1.70
JNUDuoEvo99.27 ± 0.0597.22 ± 1.0488.79 ± 5.2895.09 ± 2.10

These results evaluate the complete DuoEvo framework against the frozen curriculum, capturing the combined effects of adaptive challenge generation, parameter learning, and conditional architectural adaptation.

Evaluation at −7/−8 dB using the same checkpoints without retraining yields gains of 2.25/3.25 pp on PU, 0.89/0.46 pp on CWRU, and 3.40/3.85 pp on JNU over the frozen curriculum.

W2 & Q2: How stable is the structural contribution, and what explains it?

All seven trajectories that underwent architecture replacement have higher final HN BA than Evolve-NoStruct; all five that retain replacement architectures also improve, with a mean gain of 1.03 pp.

The table summarizes round-20 HN BA (%) from the original manuscript, Appendix Table 11, for trajectories that underwent architecture replacement; gains are DuoEvo − Evolve-NoStruct (pp).

Dataset / seedDuoEvoEvolve-NoStructGain (pp)
PU / 1196.5894.74+1.84
PU / 2397.1796.37+0.80
PU / 3796.2695.68+0.58
CWRU / 1198.2196.59+1.62
CWRU / 3797.9896.87+1.11
JNU / 2396.1695.32+0.84
JNU / 3796.0893.11+2.97

CWRU seed11 and JNU seed37 later returned to the initial architecture; their endpoints reflect the complete replacement-and-rollback paths.

LLM proposals, candidate training, and curriculum evolution make gain magnitudes trajectory-dependent; validation gates, switch audits, and patience rules control replacement and search termination.

Appendix E's Anchor-Retrain analysis further estimates architecture-path effects of +1.65/+0.10/+1.45 pp on PU/CWRU/JNU, with paired 95% intervals [−1.61, 4.92]/[−0.33, 0.53]/[−4.79, 7.69], including subsequent training interactions.

W3 & Q3: Do gains justify search cost, and are optimization settings comparable?

We report development costs and added schedule comparisons to clarify the performance–cost tradeoff. Inference requires no additional search. The accuracy comparison matches 648 deployment-training epochs; total OEE also counts screening, confirmation, rebuilds, and audits.

DatasetDeployment OEETotal training OEERecorded elapsed hoursLLM calls
PU6483,320–4,58424.83–36.34480–560
CWRU6482,544–4,1923.38–6.11440–520
JNU6482,544–4,7443.54–9.85440–560

Structural search accounts for 74.5–86.3% of training OEE. OEE counts training epochs; elapsed hours include API waits and machine-load effects. ClassBD and applicable evolved models retain auxiliary losses; MART+LORE disables them.

Full DuoEvo carries weights forward with round-wise cosine restarts (8 bootstrap + 20×32 epochs). Frozen curricula use continuous cosine. The added fixed-ClassBD comparison tests restart/continuous cosine at peak LR 0.01, with 320 proposals, common augmentation/validation rules, and segment-boundary momentum resets. Its online variant uses revised coverage/progress selection. Validation seed 97 selects continuous cosine for frozen training on CWRU/JNU and the JNU online variant, and restart for the CWRU variant. At reporting seed 11, online-variant/frozen HN BA is 96.52/97.01% on CWRU and 94.13/94.71% on JNU; this variant is separate from full DuoEvo.

W4 & Q4: Do confirmation reuse and switch audits satisfy the final deployment check?

Our branch audit clarifies the deployment non-degradation condition: the final checkpoint must pass the same-round gate comparison against the continued incumbent.

Dataset/seed/roundBranchGate-mean changeClean changeMinimum family change
PU/11/3Confirmation-weight reuse+1.5920.000−0.270
JNU/37/3Confirmation-weight reuse−0.2670.000−1.900
CWRU/11/5Restore anchor after switch audit−1.044−0.147−5.930
JNU/37/6Restore anchor after switch audit−0.3030.000−3.050

The last three original deployments fail gate-mean non-decrease, so the guarantee does not cover those paths. A final check would retain the continued incumbent. We will make this check an explicit proposition precondition. These trajectories have not been rerun with it; the audit and original HN results describe the original executions.

W5 & Q5: Do results persist with training-only normalization and new operating conditions?

We completed 36 controlled retraining runs (18 pairs) to assess normalization sensitivity. Each uses a fixed recorded curriculum and endpoint architecture, trained from scratch for 648 epochs. This tests normalization for those fixed curricula/architectures. The main comparison uses full-record normalization.

We use full-record normalization for offline diagnosis, where the complete unlabeled recording is available before analysis. Its mean and SD remove overall offset and standardize amplitude; one affine transform preserves relative amplitude differences across temporal segments. Statistics include later segments.

The table reports mean HN BA (%) across three seeds.

DatasetFixed curriculum and endpoint architectureFull-record statisticsTraining-only statistics
CWRUDuoEvo97.0695.71
CWRUEvolve-NoStruct97.4197.73
CWRUFrozen curriculum96.4797.15
JNUDuoEvo94.1893.30
JNUEvolve-NoStruct94.0792.79
JNUFrozen curriculum95.3795.68

Training-only statistics lower DuoEvo HN BA by 1.35/0.87 pp on CWRU/JNU and change some rankings in this controlled analysis.

Evaluation targets unseen interference under training operating conditions; cross-load/speed transfer also changes fault signals. Noise primitives model industrial interference: colored/band-limited background vibration, periodic harmonics, impact-induced resonant ringing, and intermittent bursts/modulation. Held-out swept_chirp, mult_speckle, heavy_burst, and ood_mix test frequency drift, signal-dependent multiplicative noise, heavy-tailed bursts, and their composition through separate implementations. Calibrated SNRs and reproducible realizations support controlled comparisons.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.