- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper study how RSI can be applied in the domain of table understanding and reasoning. To this end, this paper proposes a multi-agent pipeline, TabRSI, which jointly searches over pipeline topology, role-specific code and reasoning modality. Experiments over several benchmarks demonstrate the effectiveness of TabRSI over traditional CoT prompting.
Critical strengths
- This study is among the first to apply RSI in the domain of table understanding and reasoning.
- The study reveals that a single solver cannot recover structure that its input never carried, and proposes a modular multi-agent pipeline to address this issue.
- Experiments on seven public table benchmarks show the effectiveness of TabRSI over traditional CoT prompting.
Critical weaknesses
- It seems that TabRSI is categorized to workflow approaches, which have been widely studied in previous research. The ablation experimental results show that the main improvement comes from the workflow design rather than self-improvement modules.
- Baseline comparative results are missing, where only CoT prompting and RSI variants have been studied. As several approaches have also been verified as effective, it remains unclear whether RSI in table understanding and reasoning is competitive and cost-effective.
- The paper presentation is poorly organized, with few subsections, figures and details to make it readable. The abstract and the main text is hard to understand.
Questions for authors
See weaknesses part.