acceptodds
Withdrawn from ICLR 2027

ECG-RSI: Towards Recursive Self-Improvement via Feedback-Driven Data Synthesis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

10 views

Abstract

ECG multimodal language models depend on instruction data that connect waveform evidence to interpretation, while GAN- and diffusion-based generators provide complementary ways to expand signal coverage. A central challenge is to construct examples with specified ECG changes, consistent questions, and checkable evidence. We propose ECG-RSI, centered on reverse waveform perturbation and bidirectional signal–instruction synthesis. The forward route derives questions and explanations from real training-side waveforms; the reverse route translates target features or training-question distractors into constrained waveform edits, remeasures the resulting signal, and constructs supervision from the achieved evidence. Target attainment and non-target preservation jointly constrain acceptance. Two feedback loops connect this data engine to learning: construction feedback guides operator and prompt repair, while learner feedback specifies which capabilities, difficulty levels, and evidence representations to construct next. This process adjusts training data to the model's current errors and capabilities. On ECGBench, ECG-RSI raises the aggregate S7 score from 30.15 to 46.61 across four rounds, a gain of 16.46 points over the base model.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
ECG, medical llm, multimodal model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WofHOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ECG-RSI, a data-centric iterative training framework for ECG multimodal models. The forward pipeline generates question-answer pairs from measured features of real ECG waveforms. The reverse pipeline edits an existing training waveform to match a target feature change, re-measures the perturbed signal, and constructs evidence-aligned questions and explanations based on the achieved edits. A second feedback loop uses the current learner's error profile to decide which categories, difficulty levels, and supervision styles to generate in the next round. In each iteration, multiple sibling branches are trained from the same parent checkpoint, and the best-performing branch is promoted. The paper reports that the aggregate S7 score rises from 30.15 to 46.61 across four rounds, alongside waveform edit error statistics, a common-checkpoint ablation study, and a blinded review of 240 generated examples by two physicians.

Critical strengths
  1. The idea of reverse waveform perturbation for targeted instruction data construction is conceptually promising. Editing the waveform first and then regenerating supervision from the resulting signal provides a more controlled way to create feature-specific training examples than paraphrasing existing reports or reusing real-world annotations.
  2. The data construction pipeline is comprehensively engineered, with target attainment checks, non-target preservation checks, versioned repair mechanisms, replay buffers, same-parent branch comparison, and clinical expert review. The single-round ablation in Table 5 also provides preliminary evidence that reverse synthesis and feedback-guided selection add value beyond ordinary training questions at that specific checkpoint.
Critical weaknesses
  1. The core claim of recursive self-improvement lacks rigorous, isolated validation. The reported 30.15→46.6130.15 \to 46.61 gain across four rounds conflates multiple confounding factors: cumulative data scaling, repeated supervised fine-tuning, targeted synthesis, extra branch-search compute, and repeated model selection. The most directly relevant ablation in Table 5 is only a single-round comparison from one checkpoint. The paper does not provide a matched multi-round control with the same cumulative data volume, number of SFT updates, and branch-search budget but without learner-driven feedback. Without this control, the main results cannot support the title-level claim that feedback-driven recursive self-improvement drives the improvement.

In addition, the automatic construction loop appears to reuse the same delineation/measurement framework for defining measurable targets and remeasuring edited signals. Although a separate quality-review stage and physician audit are provided, these do not directly establish that the numerical waveform measurements used for construction are independently verified. The physician audit further underscores this concern: non-target preservation is 0.79 and evidence-supported explanations reach only 0.76, leaving a notable residual error rate for data advertised as verified.

  1. Several decision-critical quantities are described procedurally but are not instantiated clearly enough to reproduce the reported trajectory. Large portions of the main text and appendices are dedicated to workflow descriptions, prompt templates, and case walkthroughs, while key numerical specifications — including the regression tolerances for branch promotion, valid-output gates, stopping criteria, and some edit-specific tolerance thresholds — are scattered, stated only qualitatively, or not explicitly reported. Core innovations are buried under procedural documentation, and the logical chain connecting method design, experimental evidence, and conclusions is consistently unclear. The overall presentation reads more like engineering process documentation than a focused scientific contribution.
Questions for authors
  1. Can you provide a multi-round baseline control with the same cumulative number of new examples, same replay policy, same number of SFT updates, and comparable branch-search compute budget, but using a fixed or random data recipe instead of learner-driven feedback? This control is necessary to disentangle the effect of RSI from plain data scaling and repeated training.
  2. Please state explicitly how VfbV_{fb}, VselV_{sel}, and the seven final benchmarks are separated. Were any final-benchmark scores or benchmark-level error summaries used for recipe design, model promotion, stopping criteria, or hyperparameter decisions?
  3. Were the 240 physician-reviewed examples sampled from the final accepted training pool after all automatic quality gates? Also, is post-edit verification performed by the same delineator/measurement pipeline used to define the edit parameters, or by an independent verification module?
  4. Much of the paper and appendix is occupied by workflow descriptions, prompt templates, and case walkthroughs. Could you clearly state what the core technical contributions of this work are relative to prior work, and which parts are standard engineering implementation?
Reviewer_7npTOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes ECG-RSI, a framework for improving ECG multimodal language models through iterative synthetic data generation and model refinement. The key idea is to connect ECG waveform synthesis with learner feedback. Experimental results across seven ECG-related benchmarks show that ECG-RSI improves the performance.

Critical strengths
  • The authors applied the currently popular Recursive Self-Improvement technique to ECG.
  • The ablation studies are well designed.
Critical weaknesses
  • I find the motivation of applying Recursive Self-Improvement to ECG not convincing.
  • I also find that many key details of the proposed Recursive Self-Improvement method are missing.
Questions for authors
  • Could the authors clarify how ECG-RSI differs from the RSI techniques applied in other domains? What is the unique challenge related to applying RSI to ECG signals?
  • Could the authors clarify why you can make constrained changes to an existing waveform, and how can you guarantee that the changed waveforms still preserve their key properties? Are there real doctors or clinicians involved in examining and approving those changes?
Reviewer_cBLtOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents ECG-RSI, a data-centric recursive self-improvement framework for ECG interpretation. It combines forward synthesis, which generates question–answer pairs from real ECG waveforms, with reverse synthesis, which edits waveforms according to target features or distractors from training questions before constructing supervision. Signal remeasurement checks target attainment, preservation of non-target features, and consistency between waveform evidence and supervision. Construction feedback repairs the generation pipeline, while learner feedback adjusts the categories, difficulty levels, and evidence representations of subsequent training data. Candidate training runs are compared to select updates. The reported aggregate S7 score increases from 30.15 for the initial model to 46.61 for the final model.

Critical strengths
  1. Waveform-grounded and verifiable data construction. The method goes beyond textual question generation by applying constrained edits to ECG signals and deriving supervision from post-edit measurements. Checking both target attainment and non-target preservation helps reduce inconsistencies between the ECG image and its associated answer.

  2. Empirical gains supported by component analysis. The paper reports improvements across multiple ECG benchmarks and includes ablations on data construction and feedback-based selection, providing evidence beyond a comparison of initial and final model performance.

  3. Separation of sample validity from training utility. Construction feedback assesses whether an example is valid, whereas learner feedback and candidate training comparisons assess whether it improves the model. This distinction avoids equating generation quality or error frequency with instructional value.

Critical weaknesses
  1. Limited conceptual novelty in the general self-improvement framework. The main loop—evaluating the learner, identifying weaknesses, synthesizing targeted data, and retraining—is closely related to existing feedback-driven synthesis, self-training, and curriculum learning. The clearer contribution lies in ECG-specific data construction; the methodological advance over general RSI approaches needs sharper differentiation.

  2. Feedback characterizes error patterns more clearly than their causes. Category-, task-, difficulty-, and confusion-level feedback identifies where performance is weak, but does not necessarily establish whether failures arise from waveform perception, measurement, medical knowledge, or reasoning. Consequently, it remains unclear how reliably the proposed data interventions address the underlying causes.

  3. Local measurement validity does not establish full physiological plausibility. Remeasurement can verify intended feature changes, but local edits may also affect cross-lead relationships or other diagnostically relevant properties. Target and preservation checks are useful, yet their coverage of complex abnormality combinations and overall physiological consistency requires clearer justification.

  4. Performance gains need stronger compute-matched assessment. Each round involves data generation, validation, and training multiple candidate models. Improvements should therefore be assessed alongside cumulative training, historical replay, and candidate-search costs. Comparisons against continued fine-tuning, random data expansion, and simple error-based resampling under the same total compute budget would better establish the value of the recursive feedback mechanism.

Questions for authors

What are the core contributions of ECG-RSI to the general self-improvement mechanism beyond existing feedback-driven data synthesis and iterative training methods? How can these contributions be distinguished from the ECG-specific waveform editing and verification designs?

Does ECG-RSI outperform continued SFT with a fixed data strategy under the same total compute budget? Could the authors provide a controlled comparison that accounts for training all candidate branches, including discarded ones, as well as data generation, verification, and evaluation costs?

How much of the tree strategy’s advantage over chain updates comes from feedback-guided data selection, and how much comes from additional candidate training and selection? Could experiments matching the number of candidates and total compute budget disentangle these factors?

Could the authors provide direct evidence linking identified weaknesses, targeted data generation, and corresponding capability improvements? For example, does generating data for a particular error category significantly reduce those errors on an independent test set, and how does it affect performance on other categories?

What specific algorithms or tools implement reverse waveform perturbation? Which operations are predefined by humans, and which are generated or modified by the model? For complex abnormalities, how are cross-lead consistency and physiological plausibility verified after editing?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.