- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes V-CUE, which uses image generation models to provide visual hints for answering cultural questions. It first performs Uncertainty Detection (UD). If the LLM is uncertain, it extracts the cultural region, object, and symbol from the question as conditions for a diffusion-based image generation model (VG). Finally, Visual-based Generation Reconstruction (VR) uses the original question and generated image to produce the final answer for multimodal LLMs. For text-only LLMs, a vision-language model first generates a caption of the image, which is then combined with the original question to generate the answer.
Critical strengths
- The overall pipeline is easy to understand.
Critical weaknesses
- Including CulFiT in Table 1 is not meaningful without evaluating Qwen2.5-7B and Llama3.1-8B with V-CUE, as there is no directly comparable baseline.
- Figure 3 lacks a color legend. I can only infer that purple, green, and orange represent LLM-VL, LLM-reasoning, and LLM-text systems respectively based on Table 1.
- Section 6.3 does not explicitly identify the models discussed, requiring readers to refer to the table. It also does not justify why different rows are selected for different comparisons. Since the results show consistent trends, an overall analysis would be clearer. Otherwise, the authors should explain why you select these results to compare.
- In Section 6.4, UD-gated achieves the lowest accuracy. To demonstrate a favorable accuracy–efficiency trade-off, the authors should also include the original model without V-CUE as the fastest baseline.
- Introducing an additional image generation model may introduce the knowledge from its, producing more comprehensive information. Thus, the improvements do not enough to demonstrate that visual representations could provide more information. Section 6.1 also shows that stronger image generators lead to better downstream performance.
- The paper repeatedly refers to "supplementary material," but none is provided. If the authors mean the appendix, they should specify the corresponding sections.
Questions for authors
- In Section 3, what is the difference between (Q) and (C)? If they refer to the same thing, please use consistent notation.
- Why are DeepSeek-V3/R1 and GPT-3.5-turbo/o3-mini used to distinguish LLM-text and LLM-reasoning systems? A more consistent comparison would use the same reasoning model with reasoning disabled (DeepSeek-R1) or reduced thinking effort(o3-mini), as the authors done with Qwen3-32B and Qwen3-235B-A22B. Additionally, why not include reasoning VL model to form a 2×2 comparison?
- In Table 1, CultureLLM and CulturePark require training, but their original papers did not use Qwen3-32B. How were these methods reproduced? Did the authors follow the original training setting while only replacing the backbone model? Also, why not apply V-CUE to these models? Although you describe them as “training-time reference points”, V-CUE is an inference-time method and should still be applicable for them.
- In Table 1, enabling reasoning reduces the accuracy of both Qwen3-235B-A22B and Qwen3-32B. Are these really generate wrong answers, or do the models reach the maximum token length before finishing? The max token length is not specified in the experiment setting. If it is limited to around 8,192 tokens or fewer, the reasoning models may simply exceed it. While, V-CUE may simply help models answers with lower token length rather than help the model answer correctly.
- In Section 6.3, why is the performance drop without UD attributed to " LLMs retain limited inherent capability for cultural attribute recognition"? Doesn't it simply show that UD filtering is useful? One possible explanation is that generating images without filtering introduces additional noise. However, this is only a hypothesis and requires further verification. Moreover, if the LLM has limited ability to recognize cultural elements, as you concluded, why do you use the same model to extract culture-aware condition units?
- In Section 6.3, could the authors replace generated images with retrieved similar images as a VG ablation to demonstrate the benefit of image generation? Also, when VG or VR is removed, what role does UD still play, given that it originally determines whether visual generation should be triggered?
- In Section 6.3, what exactly does "removes VR for text/reasoning systems" mean? Without VR, how is the final answer generated, and how does the language model use the image produced by VG?
- What is the difference between "Always-on" in Section 6.4 and "w/o UD" in Section 6.3?
- Which specific models are used in the experiments in Section 6.5?