- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes to train MLLM to answer questions about combustion simulation. The authors benchmarked several MLLMs which show chance performance. To resolve it, they generates several pair scene-QA data and SFT the Qwen models which result in significant increases.
Critical strengths
- This paper is clearly written and easy to follow. The motivation and concepts are delivered clearly.
- Several models are benchmarked making the evaluation more sound.
Critical weaknesses
- Misleading title: The "Real-Sim-Real" statement usually means real-to-sim reconstruction or generation and then real-to-sim refers to retargeting or domain adaptation, and neither is relevant to the scope of this paper, which focuses on VQA. Thus, this paper reduces to a domain-specific VQA problem focusing on enabling VQA for combustion-relevant information. At the same time, the technical novelty is relatively small and to the reviewer, the main contribution is the dataset, whose size is relatively limited and small.
- Train-val-test split: In L297 the authors mention obtaining 67k scene-clip pairs, and then split to 7:2:1 (train/validation/test), without mentioning if the validation and testing sets are held-out or not. From the context, it seems that the clips are not held out (7 scenarios and 228 cases), making the results less convincing as the testing data may have leaked to the model during training. Can the authors elaborate more about the dataset split setup?
- Base models' performance are chance: In Table 1, most performance are likely ~25%, even for GPT-5.5 and Gemini-3.5 the best accuracies are just 33%. Can the authors describe and specify how the inputs are provided to the model?
- When do we need this benchmark? The use case assumes access to the field simulation, and when one ran the simulation, the simulation did not have the problem of consistency or ordering problem. In real-world, it is hard to think of a need that we require MLLM to answer such questions and so it is hard to imagine a use case when an MLLM has to answer the coherence, selection, and R2S tasks.
Questions for authors
I have all the questions listed above. It would be helpful to elaborate more about the setup, the contribution, and the title naming.