- Rating
- 1(−1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(−1)
Read review
Paper summary
The authors introduce the use of visual tokens as a representation compression mechanism used to rerank retrieval results in RAG systems in QA applications. Experiments are performed on reference text sets in QA tasks.
Critical strengths
They propose a representation compression mechanism in QA systems. Compressing relevant information in a conversation with a natural language engine is essential to keep the context as complete as possible to ensure the quality of the conversation.
Critical weaknesses
The authors sell the use of visual tokens as a "natural" mechanism for token representation compression, but there is no real estimate of the compression rate achieved with the visual representation. In the experimental evaluation, the average number of tokens used according to different experimental configurations has been reported, but it is not shown the average number of textual tokens (as a reference). It would be useful to shown the actual compression rate of visual tokens.
In this direction, instead of using only Qwen3-Reranker-4B as the reference teacher, the different textual baselines "teacher" should be compared with the version with visual token.
The compression due to the visual token is due to the resolution of 96 DPI. This value is not a realistic resolution in a scenario of text recognition in document images (which requires a minimum of 150DPIs to have reasonable OCR results), so it is even more relevant to assess the impact of the compression factor of the visual token in relation to textual tokens usiing realistic DPIs values.
In this sense, instead of extracting visual tokens from an image generated by an arbitrary text rendering mechanism, the authors could have learned these "visual" tokens from an implicit representation of without the need for an explicit rendering of text and aligned it with the teacher of the Eq. (1)
Questions for authors
-
In line with the elements discussed in the previous section. What is the real compression factor of visual tokens in relation to textual tokens?
-
How to apply the proposed scheme to images of scanned documents at normal resolutions (from 150 to 300 DPIs)
10 comments
Response to Reviewer Vj8Y (2)
W3/Q2
In our setup, 96 DPI is a rendering parameter that converts font sizes from points to pixels, not a scanning resolution. At 96 DPI, a 12pt font corresponds to 16 pixels, and documents are rendered at a width of 896 pixels and a maximum height of 896 pixels. Qwen's visual preprocessing operates on pixel values and image dimensions; changing only the DPI metadata while keeping the pixel array unchanged does not alter the model input or the number of visual tokens. Rendering directly from source text allows us to control character size in pixels and information density to balance performance and token efficiency. The rendering experiments in the paper support this design choice, which is distinct from DPI requirements for scanned documents. Scanned documents at 150–300 DPI constitute a different input setting from the text rendering studied here.
W4
Learning compressed latent representations directly from text is a valid alternative, but it addresses a different research question. RenderRank investigates whether an existing VLM's ability to represent text visually can be used for document relevance scoring without jointly training a new compressor. We therefore freeze the visual encoder and feature mergers and train the language model using Cross-Modal Relevance Distillation and Query-Local Relevance Discrimination.
Explicit rendering serves a specific purpose: it converts text documents into inputs that the existing visual encoder can process and allows input length to be controlled through rendering density. Replacing this step with a learned latent compressor would change both the document representation and the components being trained. RenderRank does not rule out that alternative; it offers a distinct approach that reuses existing visual representations to make more document content available for relevance scoring within a limited input budget.
We hope that these responses and additional results clarify the scope of our work and address your concerns.
I thank the authors for their responses. However, I believe my main concern has not been addressed: the visual compression factor appears to be determined by the image resolution, so the reported compression stems largely from the initial choice of rendering at 96 DPI (16 px characters). My questions were intended to help clarify these doubts; unfortunately, the responses have raised more questions than they have answered.
-
Q1 (compression factor). The 32.4% reduction includes the query and the instructions, and therefore does not measure the compression of the document itself, as requested in my original question. This is confirmed by the variability across datasets (19–41%), which grows with document length. The only document-level figure appears in the response to another reviewer (XeLq, Q3) and corresponds to a single TREC-COVID document. This value should have been reported for all datasets and at different resolutions (DPI).
-
Q2 (resolution). The distinction between rendering DPI and scanning DPI is not a substantive one: in both cases, DPI sets the mapping between physical size and pixels, and 12 pt at 96 DPI is a low resolution compared with those typical of scanned documents (150–300 DPI). Since the number of visual tokens grows roughly with pixel area, moving from 96 to 300 DPI without downscaling would multiply this area by ~10, and the current compression could turn into an expansion.
Response to Reviewer Vj8Y (1)
We appreciate your recognition of the value of representation compression and your positive assessment of our methodology. Below, we provide additional results on token compression and teacher choice, and address concerns about rendering resolution and document representations by clarifying the distinction between rendering text and recognizing scanned documents.
W1/Q1
The table below reports token counts and reductions for each dataset, comparing text inputs with images rendered at the default 12pt font size using the same fully trained RenderRank model.
Image inputs reduce the number of input tokens on every dataset, with the average input length decreasing from 429.03 to 290.07—a 32.4% reduction. These counts cover the full input, including the query and instructions. For image inputs, we count both text and visual tokens.
W2
The training approach is effective with a different teacher family and with self-distillation. We used either Llama-Nemotron-Rerank-1B-v2 or the initial Qwen3-VL-Reranker-2B with text inputs as a frozen Stage 1 teacher, retaining the same student initialization and Stage 2 training procedure. The table below reports NDCG@10 across 11 BEIR datasets for the text-input reference models and the visual-input students after both training stages.
The final model trained with the Nemotron teacher achieves an average NDCG@10 of 54.29, improving on the initial model's image-input score of 50.20 by 4.09 points and reaching performance comparable to the teacher's 54.27. The self-distilled model achieves 54.80, exceeding the initial model by 4.60 points with image inputs and 0.59 points with text inputs. These results show that effective transfer of relevance signals to visual inputs is not limited to the original 4B Qwen teacher: both a teacher from a different model family and the initial model's own text-based relevance signals support high reranking performance.