- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper focuses on real estate enterprise risk attribution: how policy and macroeconomic changes may lead to financial risks for real estate.
The proposed solution is an LLM-based pipeline, which separate agents analyze different sources of evidence, an attribution agent combines their outputs, and feedback is used to revise its prompt.
My main concern is that the methodological contribution remains limited. Also, I find the description of prompt revision procedure is incomplete.
Critical strengths
-
This paper focuses on a practical problem;
-
The authors proposed a technically-reasonable method;
-
The authors conducted many experiments to prove it.
Critical weaknesses
-
The proposed RERA is a LLM-based pipline. The paper does not clearly identify a methodological insight beyond integration of existing techniques. I think the authors overstated ``textual gradient optimization''. Earlier papers, including Automatic Prompt Optimization with “Gradient Descent” and Beam Search and TextGrad: Automatic “Differentiation” via Text, make clear that “gradient” is an analogy. RERA uses the term as if it were a literal part of the method, but it does not calculate gradients or change the model’s weights. It uses feedback to rewrite a prompt. The authors should say this more plainly and avoid making the method sound more technical than it is.
-
Every metric in Table 1 measures similarity to expert reference reports; these metrics cannot independently establish factual correctness or the validity of the proposed causal chains. RERA uses multiple GPT‑4o agents and several rounds of prompt optimization, whereas the reported GPT‑4o baseline appears to be a single model run. A baseline with comparable inference and optimization budgets is needed. In Table 3, removing prompt optimization reduces BERTScore from 0.9226 to 0.8583, below the standalone GPT‑4o baseline of 0.8853 in Table 1. The reported improvement may depend heavily on prompt optimization, and the authors must explain the real contribution of multi-agent systems. Also, the appendix provides representative prompt templates, but important implementation details remain unspecified. For example, after Eq. (5), the paper says that the average score affects how many words can be changed, but it does not explain how that number is chosen. The prompt in the appendix simply takes this number as an input, without saying where it comes from.
Questions for authors
Please refer to the ``Critical Weaknesses'' part.