- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces IDEAGENT, a multi-agent framework that tracks idea evolution through lineages. They jointly drive Quality using multi-objective feedback for repair and refinement, while Diversity is achieved through lightweight sequential memory and explicit comparison against completed ideas, historical variants, and rejected proposals. To evaluate this joint objective, this paper also develops Yield, a metric that computes the largest mutually diverse set of ideas satisfying predefined quality thresholds.
Critical strengths
It introduces IDEAGENT, a multi-agent framework that tracks idea evolution through lineages using multi-objective feedback for repair and refinement, while Diversity is achieved through lightweight sequential memory and explicit comparison against completed ideas
Critical weaknesses
The evaluation relies almost entirely on LLM judges, without human expert validation. The agreement between the two external judges is also relatively low, especially for soundness (κ = 0.268), which is a key component of Yield. This raises concerns about the reliability of the reported improvements.
The proposed framework mainly combines existing techniques, including multi-agent evaluation, sequential memory, and iterative refinement. While the Quality-Diversity formulation is interesting, the technical contributions beyond existing research idea generation methods are relatively limited.
Questions for authors
Most baselines are simplified generation strategies rather than existing research idea generation systems. Although the authors provide additional scaling experiments, comparisons with stronger methods under matched inference costs are still missing.
The paper does not evaluate whether the generated ideas are actually novel with respect to existing literature or whether they can be successfully implemented. As a result, improvements in the proposed Yield metric may not translate into better research ideas in practice.