- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper investigates whether organization of information inspired by interfaces designed to support human judgment (e.g., professional trading interfaces) could also help LLMs in doing similar tasks that require it as represented by flat tables (FLAT) and panels inspired by professional trading interfaces (PANELS). For the only two LLMs the paper evaluates in its results without any strong baselines to compare with, using FLAT seems to be better scoring in cumulative attainment/E2C than PANELS and consistently across evidence budget.
Critical strengths
I do like (and see the practical value) in the particular research question that authors have aimed for, which is investigating how informative interfaces that help support professional human work (e.g. professional trading interfaces as the authors cite) could also help LLMs perform certain tasks. This alone is a very interesting RQ. However, the execution of the experiments and the level of technical rigor is not enough for ICLR. I do hope the authors could still continue this line of work and put more effort in improving the rigor of the paper before sending out to peer review.
Critical weaknesses
The paper has no substantive and definitive discussion of the results wherein the main results section is just one page (page 8). This alone merits a clear rejection of the paper. Moreover, the authors should note that Appendices are optional for reviewers to look at and not something that the authors can redirect the reviewers to at all points of the paper. The Appendix should also not contain critical details that should have been in the main content of the paper.
The paper uses only two commercial models for its main results: GPT-5.6 Luna and Claude Sonnet 5 and there are also no baseline comparisons. These two are also grounds for clear rejection of the paper for lack of technical rigor and does not meet the empirical standards of ICLR. I highly and strongly recommend the authors to read previously accepted papers from ICLR/NeurIPS/ICML to get a level of how rigorous technical experiments are conducted and accepted in these venues. Simply running your methods on two models is simply not enough, much less selecting commercial models.
Aside from the two major flaws highlighted above, as I read through the paper, I found the overall writing of the paper to be quite poor and can be significantly improved in terms of readability. I acknowledge that the authors did mention they use AI for the paper and take responsibility for it, but I still highly encourage the authors to edit out any AI written text in the main paper given that they are very poorly written and are obvious to readers.
Questions for authors
N/A
Thank you for this question. Among units where both formats were queried at the same budget and FLAT passed while PANELS failed, only the submission levels were incorrect in 7 of 11 cases for GPT-5.6 Luna and 5 of 8 cases for Claude Sonnet 5 at 40 snapshots. At 78 snapshots, all 16 Luna cases and 11 Sonnet cases had errors only in the submission levels.
We therefore tested PANELS with explicit rank labels. Adding these labels improved Luna's mean E2C AUC, bringing the mean difference within our equivalence margin. However, FLAT retained its advantage for Sonnet, including when compared with a FLAT rerun conducted closer in time. An additional experiment with Qwen also showed a remaining FLAT advantage after rank labels were added. These results suggest that explicit rank labeling can help, but that it does not consistently close the gap between PANELS and FLAT.