- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes ReArch‑Coder, a framework for requirement alignment and repository‑level architecture design optimization tailored for open‑source code models. ReArch‑Coder first calibrates the target model’s comprehension of requirement documents via diagnostic question‑answering and grounded factual hints. It then casts repository architecture design as a learnable agentic decision‑making procedure, which is optimized with GRPO using reward signals derived from requirement coverage and dependency modularity.
Critical strengths
- The problem definition is clear and practically significant. The paper targets two key failure modes of open‑source coding agents for repository‑level generation: requirement misunderstanding and poor architectural design.
- The requirement‑alignment module for the target (student) model is well‑grounded in classical software‑requirements‑engineering literature. The diagnostic question‑answering pipeline offers a principled way to detect and fix comprehension defects of the student model prior to implementation.
- The agent‑driven architecture‑design formulation is interesting. Treating multi‑step repository planning as a trainable decision trajectory optimized by GRPO with modularity‑aware rewards provides a fresh perspective for repository‑level code‑agent research.
Critical weaknesses
-
The requirement‑alignment module heavily depends on a strong closed‑source teacher model to create diagnostic questions, generate corrective hints, and check whether those hints are valid. In effect, this approach performs a form of input‑based distillation: the deep understanding of software requirements comes almost entirely from the teacher model, instead of being newly learned by the target code model. Although this workflow yields practical improvements, this component has limited conceptual novelty. It does not fix the target model’s built‑in weaknesses in understanding requirements — it only compensates for them by rewriting the input prompt.
-
The requirement‑coverage reward for GRPO optimization contains clear shortcomings. It only checks whether each requirement ID is assigned to some file in the proposed architecture. It does not check whether that file is logically appropriate to implement the given requirement. This creates an easy gaming opportunity: the model can naively attach requirement IDs to random files to inflate its reward score, while still producing poorly‑structured repository designs.
-
The experiments lack sufficient ablations over the teacher‑model component. All main results rely on powerful closed‑source teacher models for requirement alignment. The paper does not report results when substituting the teacher with weaker open‑source alternatives. Therefore, we cannot tell how much performance would drop if one wants to deploy this whole pipeline without external proprietary LLM APIs. This limits our understanding of its real‑world portability.
Questions for authors
See the weaknesses