- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies the persistence of post hoc repairs to differentiable audio models during continued training. The key observation is that a repair that is effective immediately after application may gradually be canceled by subsequent optimization, as other trainable components adapt around the intervention. The authors introduce a repair-retention metric and the notion of repair “half-life,” together with a four-arm paired-continuation protocol for evaluating whether a repair persists, is partially retained, or is “eaten.” Experiments on a DDSP-based cello synthesizer show that a fixed gain repair can lose most of its benefit after continued training, while behavioral anchoring does not consistently improve persistence.
Critical strengths
- The paper highlights an important issue that can easily be overlooked in practice: a repair that works immediately does not guarantee that it will remain effective after continued training.
- Extensive experiments are conducted to support the claim and the experimental design seems sound.
Critical weaknesses
- The paper is quite difficult to follow. A large number of terms, arm identifiers, metrics, qualifications, and experimental details are introduced very quickly. For example, C3, R3, A1–A4, B0–B6, C1–C3, and U1–U3 are all used in different parts of the paper, which makes the main narrative unnecessarily difficult to reconstruct.
- Also the abstract contains too many experimental details, numerical results, abbreviations, and qualifications before clearly communicating the basic problem, approach, and takeaway.
- The empirical scope remains limited relative to the broader framing. The main evidence comes from one differentiable audio-synthesis system and primarily a loudness/level-defect setting.
- Some conclusions are based on fairly limited statistical evidence. For example, the reported calibration-source difference of 3/3 versus 1/3 seeds is not statistically separable (p=0.20), and the authors acknowledge a confound between calibration source and initial mismatch. Similarly, the original single-seed EATEN results become considerably less stable across additional seeds.
- While the paper also claims to study how to make repairs persist, the behavioral anchoring intervention does not consistently solve the problem.
Questions for authors
- How broadly do you expect the observed repair-erasure behavior to generalize beyond the current DDSP/loudness setting? Could the authors provide experiments on another architecture, repair type, or defect to support the broader applicability of the proposed protocol and taxonomy?
- The paper notes that the 0.5/0.25 thresholds are engineering choices rather than empirically calibrated values, and the alternative CI-based criterion changes several run-level verdicts. How robust are the PERSIST/PARTIAL/EATEN conclusions to the chosen thresholds?
- Since lightweight anchoring does not reliably preserve repairs and structural reparameterization has relatively limited evidence, what is the main actionable recommendation when continued training is unavoidable?
- Could the presentation be substantially simplified?