acceptodds
Under review as a conference paper at ICLR 2027

MERID: Multimodal Exploration via Recursive Self-Improvement Agents for Major Depression Analysis

19 views

Abstract

Major depressive disorder (MDD) severely impacts daily activities and quality of life. Detecting MDD involves multimodal data, such as interview recordings and sensor measurements. This is particularly challenging, as these heterogeneous modalities often demand distinct, customized prediction pipelines. Existing efforts to address this challenge have explored both manually engineered multimodal architectures and agent-assisted pipeline development. Despite their progress, it remains challenging to autonomously revise pipelines based on experimental feedback and carry verified improvements forward into subsequent designs. To this end, we propose Multimodal Exploration via Recursive Self-Improvement Agents for Major Depression Analysis (MERID). The framework develops depression pipelines through experience-based recursive self-improvement (RSI). Grounded State Construction (GSC) grounds experience by aligning multimodal records with subject-level depression targets. Coupled Pipeline Exploration (CPE) jointly modifies representations, fusion, and predictors to build successor pipelines for classification and severity estimation. Evidence-Guided Evolution (EGE) guides revisions through feedback and verifies gains under uncertainty in small depression cohorts before inheritance. Extensive experiments on depression benchmarks show that MERID achieves the best results on multiple tasks compared with multimodal and agent-based baselines. Further analysis highlights the value of acoustic and linguistic cues for depression detection.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Agentic AI, AutoResearch, Recursive Self-Improvement, Multimodal Learning, Depression Detection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_uU9mOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper frames multimodal MDD pipeline development as recursive self-improvement (RSI). GSC builds a checked data adapter that keeps subject–label alignment fixed across feature recipes; CPE enumerates and evaluates combinations of feature recipes (sources, aggregation, fusion) and scikit-learn predictors; EGE uses multiple LLM roles (Planner/Engineer/Reviewer/Chair/Proposer) to review results, propose new components, and replace the incumbent pipeline only when the gain exceeds a bootstrap-based margin. Experiments on E-DAIC, MPDD-2025/2026 and two supplementary datasets (Mental Health Multimodal, Kaggle TESS/RAVDESS) report gains over DL (MulT, Flex-MoE, MCMoE) and LLM (Gemini 2.5 Flash, MDAgents) baselines.

Critical strengths
  1. Strong results on several MDD benchmarks. MERID clearly outperforms the reported DL and LLM baselines on E-DAIC (binary Macro-F1 0.68/0.71 vs. 0.60 for Flex-MoE; PHQ-8 CCC 0.33 vs. 0.05) and MPDD-2026 Young (PHQ-9 CCC 0.61 vs. 0.08), with consistent results across two LLM backbones (Opus 4.8, GPT-6 Astra). Gains are largest on severity estimation, where the baselines largely fail.
  2. Transparency. The appendix discloses prompts, run records, per-seed results, and negative findings (e.g., the preregistered RSI test, baseline prediction collapse), which makes the claims checkable.
Critical weaknesses
  1. LLM-assisted AutoML rather than RSI. What is improved is the configuration of traditional ML pipelines (feature recipes + scikit-learn predictors), while LLM weights, role definitions and protocol stay fixed (§3); this is close to well-known AutoML / agent-based pipeline search (Auto-WEKA, MFAS, DS-Agent, AutoML-Agent, all cited by the authors). The evidence for "recursive" improvement is also weak: main runs use a single revision round (T=1, App. B.5), the review arms do not beat random revision in the eight-round study (−0.0122 [−0.0346, 0.0003], App. C.2.1), and larger budgets/more rounds give no consistent gain.
  2. The core module (EGE) is under-specified. EGE is the main technical contribution, yet there is no algorithm box or complete workflow; its description is scattered across §3.4 and App. A/H. The only worked example (Fig. 7) raises the dev score while the test score is unchanged, so the novelty of the agent design is hard to assess.
  3. Missing ML/AutoML baselines. Only DL and LLM baselines are reported. Classical LR/SVM/Ridge/SVR baselines are described as implemented (App. F.2/F.3) but not reported in Table 1 or Table 21, and there is no same-budget AutoML comparison (grid/random search, Optuna) on the same features. Given the small cohorts, conventional ML may outperform DL regardless of the agent; indeed, the DL baselines collapse to majority-class predictions on E-DAIC and MPDD-2026 Elder (App. E).
  4. Ablation does not support the module contributions. Removing EGE (or CPE) improves results on Mental Health and MPDD-2025 Elder binary/quinary (Table 2). Moreover, the full model uses 5 seeds with an inherited pipeline while the removals use 3 seeds without one (§4.3), so the differences cannot be attributed to the modules.
  5. Limited evaluation reliability. MPDD-2026 test tracks contain only 22–23 subjects; several MPDD-2026 tasks have SD = 0.0000 across seeds (App. B.1), suggesting carried-over heads rather than search; two of the six Table 1 datasets are not MDD tasks (stress classification with a row-level split; acted-emotion proxies) yet enter the average rank; test predictions ensemble up to five candidates (§4.1), unlike most baselines.
Questions for authors
  1. Please report the classical baselines from App. F.2/F.3, and a same-budget grid/random search (or existing AutoML tool) over the same features and scikit-learn search space. How much of the gain remains attributable to the agent?
  2. Can you provide an ablation with matched seeds and initialization (inherited pipeline or not), and explain why removing EGE improves several tasks?
  3. In the main runs, how often did a REVISE round actually replace the incumbent, and did those replacements improve test performance?
3 comments

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.

W1. "LLM-assisted AutoML rather than RSI. …"

A1. We clarify that MERID is RSI as the literature defines it.

  • RSI does not require weight updates. An improvement is "an accepted and inherited state change" [1], and the Darwin Gödel Machine [2] self-improves with "frozen pretrained FMs" and a fixed exploration process. MERID inherits each accepted change.
  • MERID improves its own search space. Auto-WEKA and MFAS search a fixed, human-defined space; MERID's agent writes the adapter, recipes and classifier space and expands them from feedback.
  • The eight-round study compares proposers. All arms inherit under the same promotion rule, so −0.0122 [−0.0346, 0.0003] measures who proposes, not recursion.

W2. "The core module (EGE) is under-specified. …"

A2. Each EGE round: the Planner, Engineer and Reviewer analyze the results, the Chair keeps the pipeline or asks for a revision, the Proposer adds candidates, and a candidate replaces the incumbent only if its development gain exceeds the incumbent's SE. In main-table runs, 8 of 11 replacements raised test scores. We will add it as an algorithm box.

W3. "Missing ML/AutoML baselines. …"

A3. We added them. Classical baselines (App. F, development-selected) are below MERID on 12 of 15 tasks. Same-space search, no LLM or reference (mean test scores):

MethodMean of all 13 tasksE-DAIC binary (Macro-F1)
Grid search0.5400.459
Random search (32)0.5520.592
Optuna TPE (32)0.5450.546
MERID (Opus 4.8)0.5490.677

On E-DAIC binary they are 0.085–0.217 lower; on average they match MERID.

W4. "Ablation does not support the module contributions. …"

A4. We re-ran the ablation with matched seeds and initial state: removing EGE costs 0.217 on E-DAIC binary and 0.067 on MPDD-2025 Elder 5s ternary. Mental Health now favors the full system (+0.013); Table 2's reversal came from the mismatch. Removing CPE barely changes scores but raises LLM calls per run from 11.8 to 48.9.

W5. "Limited evaluation reliability. …"

A5.

  • Every MERID result now has a participant-bootstrap 95% interval. Test sizes (56, 23, 22) will be stated.
  • Every MPDD-2026 Young seed kept the inherited heads. Each seed searched, yet no ternary candidate gained over 0.018 on development (margin ≥ 0.066) and no binary candidate beat the inherited head.
  • Mental Health and Kaggle become supplementary tasks. MDD-only rank: MERID 2.50, Flex-MoE 2.75.
  • Grid and random search deploy the same 1-SE set. It has up to five models.

Q1. "… How much of the gain remains attributable to the agent?"

A1. The agent builds the space that same-space search reuses, and MERID's margin avoids E-DAIC binary's collapse (0.677 vs 0.459). The released AutoML-Agent and DS-Agent code (MERID's LLM, features, splits) reach 0.597 and 0.695 on E-DAIC binary (MERID 0.677) at $8.21 and $0.78 per run (MERID $0.60); all paired intervals include 0.

Q2. "… explain why removing EGE improves several tasks?"

A2. Re-run with matched seeds (0–4) and initial state (mean test scores; Elder: MPDD-2025):

MethodE-DAIC binary (Macro-F1)Mental Health (Macro-F1)Elder 5s ternary (official score)Elder 5s binary (official score)
MERID w/o EGE0.4590.9530.6180.791
MERID (Opus 4.8)0.6770.9650.6850.733
  • The inherited reference matters mainly on E-DAIC binary. Across 13 tasks (E-DAIC, Mental Health, MPDD-2025), removing it lowers E-DAIC binary from 0.677 to 0.479; the other 12 average +0.0002.
  • EGE's margin prevents the E-DAIC binary collapse. In a factorial (3 tasks, seeds 0–2), margin 0 lowers E-DAIC binary from 0.684 to 0.495.
  • Removing EGE helps only where development selection transfers. Unchecked development gains help on 5 of 45 seed-task intervals (MPDD-2025 Elder binary/quinary) and cost 0.217 on E-DAIC binary.

Q3. "… how often did a REVISE round actually replace the incumbent, and did those replacements improve test performance?"

A3. 11 times in 65 main-table runs; 8 raised the test score (mean +0.015).

In light of these responses, we hope we have addressed your concerns, and hope you will consider raising your score. If there are any additional notable points of concern that we have not yet addressed, please do not hesitate to share them, and we will promptly attend to those points.

References

[1] Y. Duan et al. The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement. arXiv:2609.11873, 2026.

[2] J. Zhang et al. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. arXiv:2505.22954, 2025.

Thank you for the detailed response and additional experiments. However, the new results reinforce my main concern. Same-space random search matches MERID on average across the 13 tasks (0.552 vs. 0.549). The one large gap, on E-DAIC binary, is attributed by your own analysis to the inherited reference and the 1-SE margin, not to the LLM-driven search. The search baselines run without that reference. DS-Agent also reaches a comparable or higher score at similar cost. Combined with the low replacement rate (11 of 65 runs) and no consistent gains from additional rounds, the evidence for agent-driven recursive improvement remains limited. I will keep my score.

Thank you for the follow-up and engaging with our response. We answer each sentence below, and the answers share one point. Our introduction states that "a higher development score does not necessarily justify retaining a revision". MERID addresses this reliability problem. The new results (Appendices B.5 and I of the revision) show that random search and DS-Agent are unstable where MERID is stable and robust.

"Same-space random search matches MERID on average across the 13 tasks (0.552 vs. 0.549)."

We clarify that this average hides how strongly random search depends on its budget and its draw, and that it searches the space MERID's agents wrote (Q1 A1). We extended it to five budgets on E-DAIC binary (mean ± SD) on 100 runs:

MethodDevelopment scoreTest Macro-F1Range
Random, 8 pairs0.6160.592 ± 0.0720.434–0.721
Random, 16 pairs0.6490.601 ± 0.0840.417–0.730
Random, 32 pairs0.6810.580 ± 0.0910.417–0.738
Random, 64 pairs0.7030.542 ± 0.0900.417–0.738
Random, 128 pairs0.7320.483 ± 0.0570.449–0.678
Grid, all pairs0.7360.459–
MERID (Opus 4.8)0.7170.677 ± 0.0150.665–0.694

On this task the largest budgets have the highest development scores and the lowest test scores, so the budget cannot be set from development data. Over the 13 tasks the mean of one run has an SD of 0.013, against 0.005 for MERID.

"The one large gap, on E-DAIC binary, is attributed by your own analysis to the inherited reference and the 1-SE margin, not to the LLM-driven search."

We see this attribution as support for MERID. The inherited reference and the margin are the recursion it proposes. RSI is "an accepted and inherited state change" [1], where the reference is the inherited state and the margin is the acceptance step. Our response (Q2 A2) shows that inheritance without acceptance is not enough, since a margin of 0 lowers E-DAIC binary from 0.684 to 0.495.

"The search baselines run without that reference."

We would like to point out that our response already reports the search with that reference. "MERID w/o EGE" in its Q2 table starts from the same inherited reference and selects by development score alone. It reaches 0.459 on E-DAIC binary, so giving the search the reference does not close the gap.

"DS-Agent also reaches a comparable or higher score at similar cost."

With 15 runs per system, DS-Agent is not higher than MERID, and MERID is far more stable and robust. The number of added runs was fixed in advance, and all 15 are reported (mean ± SD):

MethodE-DAIC binary (Macro-F1)MPDD-2025 Elder 1s ternary (official score)LLM calls / cost (E-DAIC)
DS-Agent0.614 ± 0.0990.629 ± 0.06420.5 / $0.77
MERID (Opus 4.8)0.681 ± 0.0140.661 ± 0.04610.6 / $0.58

On E-DAIC binary MERID's 15 seeds stay within 0.665–0.694, and its SD is seven times smaller than DS-Agent's. DS-Agent's runs range from 0.411 to 0.767, and its first five averaged 0.695. It also needs 1.9 times the LLM calls and ran on features that MERID's adapter aligned.

"Combined with the low replacement rate (11 of 65 runs) and no consistent gains from additional rounds, the evidence for agent-driven recursive improvement remains limited."

We respectfully read the low replacement rate differently. It is the acceptance rule at work. The agents proposed all 11 replacements, and 8 of them raised the test score (Q3 A3), so the rounds that change the pipeline improve it in most cases. Rounds without a verified gain keep the pipeline. DS-Agent overwrites its script in every iteration without such a check, and its E-DAIC binary test score moves by 0.21 within one run on average. In MERID the agents propose every change and a change is inherited only after it is verified, which is why its results stay stable.

Thank you again for your time and comments. We hope these responses address your concerns, and we would be grateful if you could reconsider your score.

References

[1] Y. Duan et al. The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement. arXiv:2609.11873, 2026.

Reviewer_h7jPOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

MERID automates multimodal depression-prediction pipeline development through subject-aligned data adapters, joint exploration of representations and predictors, and feedback-guided revisions with an uncertainty-scaled replacement rule. The manuscript reports improvements over the evaluated baselines on several classification and symptom-severity tasks, including E-DAIC and MPDD-2026 Young. The implementation details and extensive appendices are useful, particularly their reporting of negative results. However, the experiments do not adequately separate the contribution of recursive improvement from inherited models, initial search, and deployment selection. The statistical interpretation of “verified” improvements also exceeds what the implemented acceptance rule establishes.

Critical strengths
  1. Preserving subject–target correspondence across changing feature pipelines addresses a concrete source of evaluation errors. The adapter checks, subject-grouped validation where identifiers are available, and exclusion of test fields from review packets are sensible safeguards.

  2. MERID evaluates complete representation–predictor combinations and retains candidate-level predictions, execution failures, and revision histories. Appendix B.7 provides a useful example in which changing feature count has opposite effects across two representations, illustrating why isolated component screening can miss interactions.

  3. The appendices report limitations explicitly: constant baseline predictions, unsuccessful memory updates, mixed memory benefits, protocol amendments, and failed primary tests of directional evolution. These disclosures help readers distinguish observed performance from proposed explanations.

Critical weaknesses
  1. The main benchmark does not isolate gains produced by the reported improvement runs. Appendix B.1 states that runs start from carried-over references; with Opus, five MPDD-2026 result columns retain the same models across all five seeds. Appendix B.2 likewise attributes identical Young-cohort results across backbones to unchanged inherited heads. These results establish the performance of retained deployments, but provide limited evidence that the current agents improved them. The manuscript needs a clearer account of how the historical references were obtained, their prior development cost, and the incremental benefit of the new search. Small seed variability is particularly difficult to interpret when every seed deploys the same model.

  2. The baseline suite is insufficiently aligned with the pipeline-optimization claim. The paper discusses DS-Agent and AutoML-Agent but does not evaluate comparable pipeline-development systems. Classical baselines are described in Appendix F, yet their results are absent from the main comparison. Several neural implementations also perform poorly: Appendix E documents constant predictions, while Appendix F.1 states that MulT and Flex-MoE lack dedicated MPDD-2026 regression heads and instead derive PHQ-9 estimates from class midpoints. Consequently, the large CCC improvements do not establish superiority over well-tuned severity regressors. The zero-shot agents, which receive compressed feature summaries, address a substantially different learning setting.

  3. The acceptance rule provides a conservative heuristic rather than a calibrated verification of generalization gains. Equation 7 scales the threshold using the incumbent’s bootstrap score variability, rather than uncertainty in the paired candidate–incumbent difference, and does not itself control repeated adaptive selection on the same development subjects. Appendix G.4 correctly acknowledges that this estimate does not establish the uniform error bound required by Proposition 2. Proposition 1 concerns stored development scores; it cannot by itself justify population improvement. The theoretical scope also excludes the additional effects of terminal selection and ensemble deployment. The language of “verified improvements” should reflect these distinctions.

  4. The module ablations do not identify the causal contribution of CPE or EGE. Section 4.3 and Appendix B.3 state that full MERID uses inherited pipelines and five seeds, whereas the removal variants start without those pipelines and use three seeds, with different search and review costs. Removing EGE simultaneously disables review, the replacement margin, and the terminal gate. The resulting performance gaps therefore combine several interventions. The one-run GSC development comparisons are informative implementation checks, but offer limited evidence of improved held-out performance.

  5. Evidence for recursive self-improvement beyond ordinary iterative pipeline search remains limited. The LLM weights and roles remain fixed, and research-memory learning is disabled in the main benchmark. In Appendix C.1, only five of 54 runs improve their retained development score after initialization, all at the first recorded revision step; evolving memory produces seven test gains and seven losses. The eight-round study supplies a useful random-revision comparison, but all evolving-minus-random test-slope intervals include zero. Its two primary directional-evolution endpoints also fail the prespecified criterion. These findings do not demonstrate that recursion reliably improves proposal quality or moves pipelines toward clinically grounded representations, despite the stronger contribution statement in Section 1.

  6. The clinical scope and uncertainty of the benchmark conclusions need tighter treatment. Appendix F identifies the Kaggle audio–visual task as neutral-versus-calm acted-emotion classification, while Mental Health predicts stress categories using a row-level split without participant identifiers. These are supplementary prediction tasks rather than direct evidence of MDD detection. The clinical test cohorts are also small: E-DAIC has 56 test participants, and MPDD-2026 has 23 Elder and 22 Young participants. Seed standard deviations on these fixed cohorts do not quantify uncertainty across patients. The ethics statement acknowledges these limitations, but the headline depression framing and performance discussion should distinguish the evidence more consistently.

Questions for authors
  1. Report the initial reference, post-initialization model, final retained model, and deployed ensemble separately for each task. What performance is gained at each stage, and how much historical search produced the starting reference? Clarify whether reference construction and protocol choices were fixed before the corresponding test results were inspected.

  2. Run paired component comparisons from identical initial pipelines, candidate spaces, folds, and deployment rules. Separate feedback-guided proposals, the acceptance margin, the terminal gate, and ensembling. Include both cold-start and inherited-reference settings.

  3. Compare against a pipeline-development baseline and budget-matched random or conventional optimization over the same search space. Report the classical baselines already described in Appendix F and include directly trained PHQ regressors.

1 comment

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.

W1. "The main benchmark does not isolate gains …"

A1. Inheritance is part of the method: a pipeline is replaced only if a candidate's development gain exceeds the margin. The MPDD-2026 Elder ternary head was replaced in every seed; on Young, no candidate gained over 0.018 (margin ≥ 0.066), so both heads were kept.

W2. "The baseline suite is insufficiently aligned …"

A2. We added the App. F classical baselines (below MERID on 12 of 15 tasks) and development-selected ridge PHQ regressors, which reach 0.225 on MPDD-2026 Young (MERID 0.605) and 0.327 on E-DAIC PHQ-8 (MERID 0.327); on Elder, ridge is higher (interval spans 0).

W3. "The acceptance rule provides a conservative heuristic …"

A3. Eq. 7 is a conservative decision rule, not a significance test. A zero margin lowers E-DAIC binary from 0.684 to 0.495, so it protects test scores. We will replace "verified" with "margin-accepted".

W4. "The module ablations do not identify the causal contribution …"

A4. We re-ran the ablation with matched seeds (0–4) and initial state: removing EGE costs 0.217 on E-DAIC binary and 0.067 on MPDD-2025 Elder 5s ternary; 11 of 45 seed-task intervals favor MERID and 5 favor removal. Removing CPE changes scores little but uses 4× the LLM calls; within EGE, the margin drives the gain.

W5. "Evidence for recursive self-improvement … remains limited. …"

A5. We clarify that MERID is RSI as the literature defines it.

  • RSI does not require weight updates. An improvement is "an accepted and inherited state change" [1], and the Darwin Gödel Machine [2] self-improves with "frozen pretrained FMs" and a fixed exploration process. MERID inherits each accepted change.
  • The eight-round study compares proposers. All arms inherit under the same promotion rule, so slope differences measure who proposes, not recursion.
  • The recursion is stable across runs. We ran a 3-generation chain from no reference (13 tasks × 5 seeds; rule fixed in advance; test read at the end; one run repeated after malformed LLM output before any test read): 32 of 130 runs replaced the inherited pipeline; mean test scores were 0.534, 0.534, 0.535. Some tasks climb: E-DAIC PHQ-8 0.306→0.342→0.342, Mental Health 0.961→0.965→0.969.

W6. "The clinical scope and uncertainty …"

A6. We moved Mental Health and Kaggle to supplementary tasks and added participant-bootstrap 95% intervals.

Q1. "Report the initial reference … and deployed ensemble …"

A1. Mean test scores, seeds 0–4:

Stage (deployed ensemble)MPDD-2025 Young 5s ternary (official score)Mental Health (Macro-F1)
Start: development-selected reference0.302none
Round 00.3090.953
Round 1, MERID (Opus 4.8)0.3380.965

Round 1 replaced the pipeline in 3 of 5 Young seeds and 4 of 5 Mental Health seeds; the retained single models score 0.353 and 0.963. Mental Health has no reference, so its gains are MERID's own. Overall, 8 of 11 accepted replacements raised test scores (mean +0.015). A run without a reference costs 12.3 LLM calls ($0.68, 8 min). The 1-SE rule was adopted after one seed-0 test comparison; all 80 reported runs then applied it unconditionally.

Q2. "Run paired component comparisons …"

A2. We crossed review on/off with margin 1 SE/0 (3 tasks, seeds 0–2, with and without the reference; 4 runs rerun after a gateway outage). With the reference, margin 0 lowers E-DAIC binary from 0.684 to 0.495. With the 1-SE margin, review off changes test scores by under 0.013; the margin carries EGE's gain. Without a reference, E-DAIC binary drops from 0.677 to 0.479 (0.197 [0.061, 0.339]); the other 12 tasks average +0.0002. Ensembling lifts E-DAIC binary from 0.499 to 0.677; the terminal gate is off.

Q3. "Compare against a pipeline-development baseline …"

A3. We ran the released AutoML-Agent and DS-Agent code (same features, splits, LLM) on E-DAIC binary / Elder 1s ternary: they reach 0.597 / 0.602 and 0.614 / 0.629, MERID 0.681 / 0.661 (last two over 15 runs); all paired intervals include 0. Same-space grid, random and Optuna search (no LLM) reach only 0.459–0.592 on E-DAIC binary and are within 0.009 of MERID on average.

In light of these responses, we hope we have addressed your concerns, and hope you will consider raising your score. If there are any additional notable points of concern that we have not yet addressed, please do not hesitate to share them, and we will promptly attend to those points.

References

[1] Y. Duan et al. The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement. arXiv:2609.11873, 2026.

[2] J. Zhang et al. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. arXiv:2505.22954, 2025.

Reviewer_N8pqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Major depressive disorder (MDD) analysis often contains multimodal data including multiple sensors and interview recordings. Existing methods mainly use fixed and manually designed prediction pipelines from data processing, feature extraction to prediction. Therefore, this paper proposes MERID to explore recursive self-improvement pipelines for MDD prediction. Comprehensive experiments on existing multimodal MDD datasets demonstrate effectiveness of MERID.

Critical strengths

The paper presents comprehensive experiments and analysis to demonstrate MERID effectiveness of MDD predictions.

Critical weaknesses

Although the paper gives a motivation for automation such as heterogenous inputs require different preprocessing, fusion; the system design or research gap claims does not establish why a RSI mechanism is needed or beyond just automated/adaptive pipeline selection. And the evidence and ablation do not convincingly isolate the benefits. In the abstract, intro and related work, a lot of terms and claims are explained in the latter part of the paper which make this challenging to read and to understand the research gap.

Questions for authors

What distinguishes MERID’s recursive self-improvement from iterative pipeline optimization?

The introduction acknowledges that DS-Agent and AutoML-Agent already use experimental feedback to refine pipelines. Which specific capability is missing from these approaches, and which experiment isolates the benefit of MERID’s proposed solution?

1 comment

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.

W1. "… does not establish why a RSI mechanism is needed or beyond just automated/adaptive pipeline selection."

A1. RSI is needed because one-shot pipeline selection overfits small clinical cohorts: on E-DAIC binary, the best development pipeline (dev 0.736) reaches only 0.459 on test. Two ingredients of RSI, as defined by Duan et al. [1], prevent this:

  • Inheriting the accepted state avoids this collapse. Each round starts from the accepted pipeline, candidate space and experience, not from scratch. Without the inherited pipeline, MERID reaches 0.479 on E-DAIC binary; with it, 0.677.
  • The margin stops overfitted changes from being inherited. A change's development gain must exceed the incumbent's subject-level bootstrap SE (Eq. 7). With the margin set to 0, E-DAIC binary falls from 0.684 to 0.495 (seeds 0–2).

W2. "And the evidence and ablation do not convincingly isolate the benefits."

A2. We re-ran all variants with matched seeds (0–4) and initial state (mean test scores):

MethodE-DAIC binary (Macro-F1)MPDD-2025 Elder 5s ternary (official score)
Grid search (no LLM)0.4590.638
MERID w/o EGE0.4590.618
MERID w/o CPE0.6810.671
MERID (Opus 4.8)0.6770.685

EGE adds 0.217 on E-DAIC binary and 0.067 on MPDD-2025 Elder 5s ternary; against w/o EGE, 11 of 45 seed-task intervals favor MERID and 5 favor the variant (all Elder binary/quinary). CPE changes scores by 0.006 on average (9 tasks) but cuts LLM calls per run from 48.9 to 11.8.

W3. "… a lot of terms and claims are explained in the latter part of the paper …"

A3. GSC (Grounded State Construction) builds a checked adapter that keeps records aligned with subject labels; CPE (Coupled Pipeline Exploration) trains and scores feature recipes and predictors jointly; EGE (Evidence-Guided Evolution) reviews the results and replaces the incumbent only if the gain exceeds a margin; RSI means each accepted change is inherited by the next round. MERID fills a gap: MDD pipelines are hand-designed over a fixed space, while unchecked search overfits small clinical cohorts. The revision defines them at first use.

Q1. "What distinguishes MERID’s recursive self-improvement from iterative pipeline optimization?"

A1. Iterative pipeline optimization keeps the best-scoring pipeline of each round. In MERID, the accepted state, including the candidate space the agent expanded, is where the next round starts, and only changes whose gain exceeds the margin are inherited. This is RSI under Duan et al. [1]: an improvement is "an accepted and inherited state change".

Q2. "… Which specific capability is missing from these approaches, and which experiment isolates the benefit of MERID’s proposed solution?"

A2. AutoML-Agent and DS-Agent are general-purpose ML agents; MERID is an RSI system for clinical prediction. They lack two of its capabilities, and setting the margin to 0 isolates the second:

  • MERID trains and scores every candidate on subject-aligned data (Eq. 2). AutoML-Agent ranks plans by training-free LLM estimates.
  • MERID inherits only changes whose gain exceeds a margin (Eq. 7). AutoML-Agent checks printed test metrics, and DS-Agent keeps a solution when test performance improves (Alg. 1). Setting only the margin to 0 (3 tasks, seeds 0–2) lowers E-DAIC binary from 0.684 to 0.495.

We ran the public AutoML-Agent and DS-Agent code with MERID's features, splits and LLM (updated: DS-Agent and MERID 15 runs, AutoML-Agent 5; test labels outside the sandbox; mean ± SD):

MethodE-DAIC binary (Macro-F1)MPDD-2025 Elder 1s ternary (official score)LLM calls / cost, E-DAICLLM calls / cost, Elder
AutoML-Agent0.597 ± 0.0390.602 ± 0.05763.2 / $8.2123.2 / $2.61
DS-Agent0.614 ± 0.0990.629 ± 0.06420.5 / $0.7720.5 / $0.79
MERID (Opus 4.8)0.681 ± 0.0140.661 ± 0.04610.6 / $0.5811.9 / $0.65

MERID's means exceed AutoML-Agent's on both tasks (all five E-DAIC seeds higher) at 1/14 and 1/4 of its LLM cost. DS-Agent (development scores for retention) is not higher than MERID, needs 1.9× and 1.7× its LLM calls and is far less stable on E-DAIC (runs 0.411–0.767 vs 0.665–0.694). With 56 and 80 test participants, all paired intervals include 0.

In light of these responses, we hope we have addressed your concerns, and hope you will consider raising your score. If there are any additional notable points of concern that we have not yet addressed, please do not hesitate to share them, and we will promptly attend to those points.

References

[1] Y. Duan et al. The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement. arXiv:2609.11873, 2026.

Reviewer_mHFBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MERID, an LLM-agent framework that builds multimodal depression detection pipelines by recursive self-improvement, proposing and revising feature recipes and scikit-learn predictor configurations based on development-set feedback. Grounded State Construction keeps multimodal records aligned with subject-level targets, Coupled Pipeline Exploration searches representation, fusion and predictor choices jointly, and Evidence-Guided Evolution uses multiple reviewing agents to guide revisions and accepts a new pipeline only when its gain exceeds a bootstrap uncertainty margin. Experiments on E-DAIC, MPDD-2025, MPDD-2026 show that MERID achieves the best results on multiple tasks compared with multimodal and LLM-based baselines.

Critical strengths

The three modules are well-motivated and the ablations suggest that each module contributes to the performance

Critical weaknesses

As I understand it, MERID performs hyperparameter tuning for ML models, which a standard grid search can also do without LLM or LLM agents. I would therefore suggest adding a grid search baseline. It should use the same search space, the same data splits, and the same selection rule, and it should not inherit any pipeline. For both approaches, please report test performance together with LLM cost, compute, and human effort. This comparison would make the benefit of the agentic search much clearer.

It would also help to clarify how MERID is positioned relative to LLM-based hyperparameter and pipeline optimization methods, since its loop similarly has an LLM propose scikit-learn configurations from development scores. Related agents such as DS-Agent and AutoML-Agent are cited, but a short discussion of how MERID differs, and ideally an empirical comparison, would strengthen the contribution.

Questions for authors

See above

1 comment

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.

W1. "… MERID performs hyperparameter tuning for ML models, which a standard grid search can also do without LLM or LLM agents. I would therefore suggest adding a grid search baseline. … For both approaches, please report test performance together with LLM cost, compute, and human effort. …"

A1. We ran this baseline as specified. We clarify that MERID searches full multimodal pipelines, not model hyperparameters alone: each candidate pairs a feature recipe (modalities, features, aggregation) with a predictor configuration, and our grid covers this whole space. For each main-table run (13 tasks, seeds 0–4), the grid searches that run's candidate space without the inherited pipeline or LLM, with the same development splits and 1-SE rule. Mean test scores:

MethodE-DAIC binary (Macro-F1)E-DAIC PHQ-8 (CCC)Mental Health (Macro-F1)Mean of all 13 tasks
Grid search0.4590.3170.9530.540
MERID (Opus 4.8)0.6770.3270.9650.549

MPDD-2025, official score (bin / ter / quin: binary / ternary / quinary; 1s / 5s: window length):

MethodElder 1s binElder 1s terElder 1s quinElder 5s binElder 5s terElder 5s quinYoung 1s binYoung 1s terYoung 5s binYoung 5s ter
Grid search0.7890.6580.4840.7830.6380.4540.4100.3510.3820.348
MERID (Opus 4.8)0.7590.6300.4740.7330.6850.4020.4030.3650.3840.338

Averaged over all 13 tasks, MERID is higher than the grid (+0.009).

Per runLLM callsLLM costWall timeHuman effort
Grid search0$06.6–26.5 minwriting the search space (here reused from MERID)
MERID (Opus 4.8)11.6$0.658.4–30.5 mina 20-line config (data path, task, metric, budget)

Wall times: means over seeds 0–2 on E-DAIC binary, E-DAIC PHQ-8 and MPDD-2025 Elder 1s ternary, same batch. A run without an inherited pipeline costs 12.3 LLM calls, $0.68 and 8 min.

W2. "It would also help to clarify how MERID is positioned relative to LLM-based hyperparameter and pipeline optimization methods … a short discussion of how MERID differs, and ideally an empirical comparison, would strengthen the contribution."

A2. AutoML-Agent and DS-Agent are general-purpose machine-learning agents that build a working solution for a given dataset (DS-Agent by adapting past Kaggle cases). MERID is a recursively self-improving system for clinical prediction: each round inherits the accepted pipeline, candidate space and evidence on which modalities hold up across participants. It differs in three ways:

  • MERID trains and scores every candidate. AutoML-Agent ranks plans by training-free LLM estimates and trains only the chosen one.
  • Test data never enter MERID's decisions. DS-Agent retains a solution when test performance improves (Alg. 1, line 16), and AutoML-Agent checks printed test metrics on a random 70/20/10 split.
  • MERID accepts only changes whose gain exceeds a margin. The incumbent is replaced only when the development gain exceeds its subject-level bootstrap SE (Eq. 7) on subject-aligned records (Eq. 2). LLAMBO, AgentHPO and SELA optimize on one table and keep the best observed result, with no subject grouping or margin.

We ran the released AutoML-Agent and DS-Agent code on two tasks with MERID's features, splits, evaluator and LLM (updated Nov 16: DS-Agent and MERID 15 runs per task, AutoML-Agent 5; test labels outside the sandbox; mean ± SD):

MethodE-DAIC binary (Macro-F1)MPDD-2025 Elder 1s ternary (official score)LLM calls / cost, E-DAICLLM calls / cost, Elder
AutoML-Agent0.597 ± 0.0390.602 ± 0.05763.2 / $8.2123.2 / $2.61
DS-Agent0.614 ± 0.0990.629 ± 0.06420.5 / $0.7720.5 / $0.79
MERID (Opus 4.8)0.681 ± 0.0140.661 ± 0.04610.6 / $0.5811.9 / $0.65

MERID scores higher than AutoML-Agent in all 5 paired seeds on E-DAIC and 3 of 5 on Elder, at 1/14 and 1/4 of its LLM cost. DS-Agent (development-score retention, TF-IDF case retrieval) is not higher than MERID; MERID uses 42–48% fewer LLM calls and is far more stable on E-DAIC (single runs 0.665–0.694 vs 0.411–0.767). With 56 and 80 test participants, all paired intervals include 0.

In light of these responses, we hope we have addressed your concerns, and hope you will consider raising your score. If there are any additional notable points of concern that we have not yet addressed, please do not hesitate to share them, and we will promptly attend to those points.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of Response

We sincerely thank all reviewers for their insightful comments and constructive suggestions!

The reviewers recognize that our modules are well-motivated (Reviewer mHFB), that subject-aligned evaluation addresses a concrete source of evaluation errors (Reviewer h7jP), that MERID achieves strong results on several MDD benchmarks, with the largest gains on severity estimation (Reviewer uU9m), that our experiments are comprehensive (Reviewer N8pq), and that our appendices and disclosures, including negative results, make the claims checkable (Reviewers h7jP, uU9m).

Following their suggestions, we have added same-space grid, random and Optuna search, classical baselines, PHQ regressors and the released AutoML-Agent and DS-Agent code (Reviewers mHFB, h7jP, uU9m), re-run the ablations with matched seeds and initial states (Reviewers N8pq, h7jP, uU9m), run a 3-generation recursion chain (Reviewer h7jP), and reported participant-bootstrap intervals (Reviewers h7jP, uU9m).

We firmly believe that MERID is, to our knowledge, the first recursively self-improving system for multimodal MDD prediction. The new results support it: on E-DAIC binary, MERID holds 0.677 Macro-F1 while same-space grid search overfits and falls to 0.459, and MERID is on par with DS-Agent and above AutoML-Agent at the lowest LLM cost. The revision will include all new results, an EGE algorithm box and definitions at first use, and we will release our code, prompts and run records, including those for the new experiments. We look forward to further discussion.

Dear Reviewers,

We’re now past the halfway point of the ICLR discussion period. Please take a moment to read the authors’ responses carefully and let them know which of your concerns have been addressed and which questions remain open. If anything needs clarification or would benefit from further discussion, please raise it and engage with the authors and fellow reviewers while there’s still time.

Thank you for your time, your thoughtful reviews, and your service to the ICLR community.

Best Regards, AC

Dear Reviewers,

The discussion period is about to close. Please read the authors’ replies to your reviews and post a brief comment on whether the clarifications affect your assessment.

Best Regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.