- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
S-ROA is a multi-turn black-box jailbreak for financial misconduct. It rewrites harmful requests into hierarchical or tabular forms that hide risk cues, using an auxiliary LLM, a judge, and a "Hidden Chain-of-Table-Thought" module. On FSBench (494 queries) across six LLMs, it reports 97.78% average ASR vs. 96.72% for RCA.
Critical strengths
Financial gray-area misuse is a real and under-studied problem. The FSBench categories could be useful if the dataset is released.
Critical weaknesses
- The method is underspecified. ReEncode, TableRep and LatentReason in Algorithm 1 are never defined, there are no prompts or examples, and Appendix B is empty. All targets are queried as text-only LLMs, so the "multimodal" claim in the title is unsupported. No ablation separates the structural transformation from RCA-style concealment.
- The evaluation is not comparable. S-ROA is judged by GPT-OSS-120B (§4.2), but baselines by a LLaMA-3.1 judge (App. A.3), and the margin over RCA is about 1 point. There is no rubric and no check that the final rewritten prompt still asks the original question. Numbers also conflict across sections: Table 2 says Qwen2.5-32B while §4.3 and Table 4 say 72B; T = 1 gives 92.35% in Table 4 but single-turn gives 44.78% in Table 6; the auxiliary model differs between §4.2 and Table 7.
Questions for authors
- Please re-score all methods with one judge and share the rubric.
- Does the final response still answer the original FSBench request?
- What does the target model actually receive? Please show a complete example.
- Where do non-text modalities enter the method or experiments?