acceptodds
Under review as a conference paper at ICLR 2027

MedScissor: Towards General and Efficient Medical Image Understanding with Vision-Language Models

Abstract

Medical vision-language models have shown substantial promise across various medical tasks. However, the large number of visual tokens generated from medical images introduces significant computational and inference overhead. Existing approaches either rely on internal attention signals, complicating their direct compatibility with efficient attention mechanisms such as FlashAttention, or are primarily designed for 3D medical imaging, limiting their applicability to broader medical imaging scenarios. To address these limitations, we propose MedScissor, a general and efficient visual token compression framework for medical vision-language models. Specifically, MedScissor first combines the semantic similarity score and neighborhood consistency score of medical visual tokens to identify redundant relationships among tokens, and then progressively merges related groups until the target compression budget is reached. Finally, MedScissor jointly considers local centrality score and global deviation score to retain one representative token with complementary information from each group. Extensive experiments demonstrate the effectiveness of MedScissor across diverse scenarios. On six representative 2D medical imaging benchmarks, MedScissor reduces the number of visual tokens by 50% while retaining 98.6% of the original performance, achieving a 1.4× prefill speedup and a 49.2% reduction in KV cache usage. On two representative 3D medical imaging benchmarks, MedScissor reduces the number of visual tokens by 70% while retaining 99.0% of the original performance, achieving a 2.1× prefill speedup and a 65.6% reduction in KV cache usage. Our code is available in the supplementary materials, and all data will be released on GitHub.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Medical VLMs, Medical Image, Token Prune

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_HrFKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MedScissor, a visual token compression framework for medical VLMs. This method is based on the local structural changes and global visual deviation of medical images. It combines semantic similarity and neighborhood consistency to progressively group redundant tokens, and then selects the original representative token according to the degree of local centrality and global deviation to retain representative and complementary visual information. Experiments on 2D and 3D medical imaging benchmarks show that MedScissor can reduce the visual token by 50-70% while retaining about 99% of the original performance and reducing reasoning overhead.

Critical strengths
  1. Sufficient comparative and ablation experiments has conducted in this paper, which show the performance of the proposed method in different scenarios and verifies the actual role of the modules.

  2. The methods proposed by this paper utilized the token similarity without additional attention score calculations, which are different with most previous medical VLM token pruning methods.

  3. The pipeline illustrated in Figure 2 is clearly to show how the methods work in a vision language model, and the algorithm section further describes the details of the methods.

Critical weaknesses
  1. The score of the token has a mathematical meaning, but it seems that it cannot be directly derived that the key medical information is retained. Normal anatomical boundaries, artifacts or other non-target areas may also have a high deviation score.

  2. The paper only provides a visualization result, and it is suggested that the author provide more comparisons of visualization results under different scenarios/different problems/modalities to further confirm the reliability of the proposed method in practical application.

  3. The discussion of related works on medical VLM and token pruning/compress omits some relevant studies, including them would make the paper more comprehensive: SeVeR[1], Photon[2]. In addition, the Introduction section uses too much boldness, which reduces the readability to a certain extent.

[1] SeVeR: Selective Visual Exposure and Retrieval for 3D Medical Image Question Answering [2] Photon: Speedup Volume Understanding with Efficient Multimodal Large Language Models

Questions for authors
  1. Is the method proposed in the article still effective for lesions with a large range, diffuse changes, low contrast or uniform internal texture?

  2. It seems that the experimental results of this method under different settings do not have consistent advantages compared with other methods. Can the author further analyze the attribution?

If the authors addressed my concerns in weaknesses and questions, I will be happy to improve my rating. I will be active in the discussion period.

7 comments
AuthorsOpenReview ↗

Sincere reply to reviewer HrFK (part1/2)

Dear Reviewer HrFK,

Thank you for your careful review and valuable constructive feedback. We have consolidated your three weaknesses and two questions into the following four issues and address them individually. We also commit to incorporating the relevant experimental results, implementation details, and clarifications into the camera-ready version.


Q1: Relationship Between Token Scoring and the Preservation of Critical Medical Information (Weakness 1, Question 1)

A1: Thank you for this important observation. We agree that global deviation scores do not directly correspond to lesion importance, as normal anatomical boundaries and imaging artifacts may also receive high scores. Importantly, MedScissor does not rely on a single score to identify lesions. Instead, it combines local semantic redundancy reduction with global information complementarity to preserve representative visual tokens.

Specifically, MedScissor first groups redundant tokens based on semantic similarity and neighborhood consistency, and then selects representative tokens by jointly considering within-group centrality and global deviation, balancing dominant semantic information and distinctive visual features. For large, diffuse, low-contrast, or internally homogeneous lesions, information preservation depends on the distinguishability of their visual features rather than lesion morphology alone.

Our experiments across six 2D and two 3D medical imaging benchmarks further support the effectiveness of this strategy at the downstream-task level. In particular, MedScissor achieves the best or highly competitive average performance across different compression ratios while maintaining high performance retention relative to the uncompressed models. These results provide indirect evidence that our method can effectively preserve task-relevant medical visual information despite substantial visual token reduction. Nevertheless, this does not constitute a strict guarantee that every lesion detail is fully preserved. We will further clarify this distinction in the camera-ready version.


Q2: Sufficiency of Visualization Experiments (Weakness 2)

A2: Thank you for this suggestion. Although the current manuscript provides limited visualization examples, our experiments cover six 2D and two 3D medical imaging benchmarks, spanning diverse imaging scenarios and medical visual question-answering tasks. These extensive quantitative evaluations provide evidence of the method's effectiveness.

We agree that additional visualizations would offer a more intuitive understanding of token preservation. Therefore, we will include further visualization comparisons across different imaging modalities, task scenarios, and compression ratios, alongside representative compression baselines, to better illustrate the practical behavior and reliability of MedScissor.

AuthorsOpenReview ↗

Sincere reply to reviewer HrFK (part2/2)

Q3: Related Work on SeVeR and Photon (Weakness 3)

A3: We appreciate the reviewer's recommendation to consider SeVeR [1] and Photon [2]. We agree that both studies are closely related to our work. The full implementations of both methods involve additional training. In particular, we have not identified a publicly available official implementation of SeVeR, making reliable reproduction under identical settings difficult.

For the publicly released Photon model, we consider adapting MedScissor to the trained Photon-S2 checkpoint by disabling the original ITS compression mechanism during inference and applying MedScissor for training-free visual token compression. The following table summarizes the reference comparison.

Table R1. Comparison on 3D Medical Imaging Benchmarks.

MethodM3D-VQA3D-RAD
Photon-S2 (w/o Compression)79.4 (0%)72.7 (0%)
Photon-S2 (ITS)81.2 (89.1%)74.6 (94.4%)
Photon-S2 + MedScissor77.8 (90%)72.3 (90%)

Values in parentheses indicate visual token compression ratios. These are provisional reference values and have not yet been validated as results from a controlled evaluation under identical settings.

The reference comparison suggests that the jointly trained Photon compression mechanism can achieve higher task performance, while MedScissor offers distinct advantages of its own. Specifically, MedScissor requires no additional training and can be directly integrated into different pretrained models, reducing adaptation and deployment costs. Furthermore, while Photon primarily targets 3D medical volumes, MedScissor supports both 2D and 3D medical imaging, offering broader applicability, as demonstrated across six 2D and two 3D benchmarks.

We will cite and discuss SeVeR and Photon in the camera-ready version and reduce unnecessary bold formatting in the Introduction to improve readability.

Reference

[1] SeVeR: Selective Visual Exposure and Retrieval for 3D Medical Image Question Answering.

[2] Photon: Speedup Volume Understanding with Efficient Multimodal Large Language Models.


Q4: Performance Variations Across Experimental Settings (Question 2)

A4: Thank you for raising this point. We acknowledge that MedScissor does not achieve the best result on every individual dataset and compression ratio. This may be attributed to differences in imaging modalities, visual structures, and image characteristics across medical datasets, which can affect how different compression strategies perform.

Importantly, our evaluation covers eight medical imaging benchmarks, including six 2D and two 3D datasets, representing diverse medical imaging types and visual understanding tasks. Despite minor variations in individual settings, MedScissor achieves the highest average performance across all three compression ratios on the 2D benchmarks and remains best-performing or highly competitive on the 3D benchmarks.

These consistent aggregate results demonstrate the effectiveness and generalizability of MedScissor across diverse medical imaging scenarios, rather than relying on a single dataset or task. We will further discuss these dataset-dependent performance variations in the camera-ready version.

AuthorsOpenReview ↗

Sincere reply to reviewer HrFK

Dear Reviewer HrFK,

Thank you again for your valuable comments and suggestions. Following your recommendations, we have completed the additional experiments and analyses (Q1–Q4) and provided detailed clarifications to address your concerns.

As you mentioned in your original review that addressing these issues would lead to a higher evaluation, we hope that our additional results and responses have sufficiently resolved your concerns. We would greatly appreciate it if you could take these improvements into consideration in your final assessment.

Thank you very much for your time and consideration.

Best regards, Authors

Reviewer_nN9gOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MedScissor reduces the visual tokens passed to a medical vision-language model without additional training. It groups tokens using feature similarity and nearby spatial information, then keeps an original token to represent each group. The selection considers how well a token represents its group and how it differs from the overall image features. The experiments cover question answering on medical images and volumes, component tests, and inference efficiency.

Critical strengths
  • The method is practical to integrate because it works on projected image features and does not need internal attention maps. Keeping representative original tokens offers a clear way to shorten the language model's input while preserving useful image information. The released code makes the grouping procedure available for inspection.
  • The experiments show a useful balance between accuracy and compression. The method achieves the best reported average performance on image benchmarks across the tested compression settings and remains competitive on volume benchmarks. The reported measurements also show faster processing of the language model's input and a smaller attention cache. Tests of connections across image slices and representative selection provide further support, although some component comparisons show little benefit.
Critical weaknesses
  • The method description does not fully match the code. Under the stated group-size rule, the requested groups may not have enough capacity to contain all tokens. The code rounds the limit up and adds a repair step. That repair can merge groups that are not spatial neighbors and can exceed the stated size limit. This does not show that the code fails, but it means the paper's strict claims about local grouping and group size need revision. The equations and pseudocode should describe the repair step and explain which properties still hold.
  • Some baseline results differ between the main and component tables, including ToMe on PathVQA and DART on MMMU-Med. These entries should be checked before making close comparisons. The efficiency claims also need a clearer boundary: the reported timing covers language-model input processing, and it is unclear whether token compression is included. Since the implementation uses CPU transfers and Python grouping, the measurements do not yet establish the same benefit for total inference time or total GPU memory.
Questions for authors
  • Could you describe the rounding and repair rules in the method, explain when repair is needed, and state which spatial and group-size properties remain guaranteed? How often does repair occur in practice?
  • Could you reconcile the inconsistent baseline entries and specify which operations are included in the timing measurements? Please separate the measured input-processing and cache benefits from claims about total inference time and memory.
4 comments
AuthorsOpenReview ↗

Sincere reply to reviewer nN9g (part1/2)

Dear Reviewer nN9g,

Thank you for your valuable and constructive feedback. We have consolidated your two weaknesses and two questions into the following two issues and address them individually. We also commit to incorporating the relevant experimental results, implementation details, and clarifications into the camera-ready version.


Q1: Group-Size Constraints, Rounding Rules, and the Repair Mechanism (Weakness 1, Question 1)

A1: Thank you for this careful observation. We agree that the current manuscript does not sufficiently describe the integer rounding rules and the repair mechanism. Specifically, we use K=⌊N(1−r)⌋K=\lfloor N(1-r)\rfloor and Smax⁡=⌈γN/K⌉S_{\max}=\lceil\gamma N/K\rceil to determine the target number of retained tokens and the maximum group size, respectively. The expression Smax⁡=γ/(1−r)S_{\max}=\gamma/(1-r) in the manuscript is a simplified formulation that omits the effects of integer rounding.

During grouping, MedScissor first performs constrained progressive merging according to semantic-neighborhood scores, strictly preserving spatial connectivity and the group-size constraint at this stage. If the target token budget cannot be reached through constrained merging, the repair mechanism repeatedly merges the two smallest groups until the target budget is satisfied.

Importantly, after repair, the algorithm still strictly guarantees that the final number of groups equals KK, every original token belongs to exactly one group, and one original token is selected as the representative of each group. However, since the repair mechanism may merge spatially non-adjacent groups, spatial connectivity and the maximum group-size constraint Smax⁡S_{\max} are no longer strictly guaranteed. Therefore, spatial adjacency and group-size constraints apply to the normal constrained merging stage, while the repair stage prioritizes satisfying the exact token budget.

To assess how frequently this mechanism is needed in practice, we additionally evaluated three 2D benchmarks (PMC-VQA, PathVQA, and VQA-RAD) and one 3D benchmark (M3D-VQA). The average proportions of samples requiring repair across these four benchmarks are reported below.

Table R1. Average Repair Trigger Rate (%).

Compression Ratio50%70%90%
Repair Trigger Rate7.2%5.1%10.1%

These results show that the repair mechanism is triggered relatively infrequently across all three compression ratios (5.1%–10.1%), indicating that the vast majority of samples reach the target budget through constrained merging alone. Thus, the repair mechanism primarily serves as a fallback to guarantee the exact token budget.

We will clarify the triggering conditions, specific operations, and constraint guarantees before and after repair, and update the corresponding equations and pseudocode in the camera-ready version to ensure consistency between the manuscript and the implementation.

AuthorsOpenReview ↗

Sincere reply to reviewer nN9g (part2/2)

Q2: Consistency of Experimental Results and Inference Efficiency Evaluation (Weakness 2, Question 2)

A2: Thank you for carefully examining the reported results. We rechecked the relevant tables in the main paper and appendix. The ToMe results on PathVQA (56.2, 55.5, and 53.1) and the DART results on MMMU-Med (51.3, 49.1, and 47.4) are identical between Table 1 and their corresponding appendix tables. Moreover, Tables 3 and 4 present component ablations of MedScissor rather than comparisons involving these pruning baselines.

Regarding inference efficiency, the LLM prefill times reported in Tables 6–9 do not include the additional token compression overhead. The operations involved in MedScissor, including graph construction, edge sorting, progressive merging, and representative token selection, take approximately 10–20 ms per sample. To provide a more complete evaluation, we additionally measured end-to-end inference latency, explicitly including the token compression overhead, and averaged the results across all eight medical imaging benchmarks evaluated in the paper, and compared MedScissor with two competitive baselines, ToMe and LLaVA-PruMerge.

Table R2. End-to-End Latency Reduction (%) Relative to the Uncompressed Model (↓ indicates lower latency).

Method50%70%90%
ToMe↓ 5.1%↓ 10.7%↓ 14.5%
LLaVA-PruMerge↓ 4.3%↓ 10.2%↓ 13.7%
MedScissor↓ 6.7%↓ 11.0%↓ 14.3%

The results demonstrate that MedScissor still achieves effective end-to-end inference acceleration after accounting for token compression overhead. It outperforms both baselines at 50% and 70% compression while remaining competitive at 90% compression.

Finally, a reduction in KV-cache memory does not necessarily translate into a proportional reduction in overall GPU memory usage. MedScissor is compatible with FlashAttention-2 and does not require storing full attention maps, resulting in relatively low and stable GPU memory usage. In contrast, FastV and SparseVLM rely on internal attention information and may incur additional memory overhead. As shown in Table 7, at 50% compression, MedScissor requires only 16.4 GiB of peak GPU memory, compared with 25.2 GiB for FastV and 28.3 GiB for SparseVLM. We will clarify the measurement scope of these efficiency metrics in the camera-ready version.

AuthorsOpenReview ↗

Sincere reply to reviewer nN9g

Dear Reviewer nN9g,

We sincerely thank you for your valuable comments and constructive suggestions on our manuscript. In response to the concerns you raised, we have conducted additional experiments and analyses(Q1-Q2), and we plan to incorporate corresponding revisions and improvements in the camera-ready version. As the rebuttal phase is coming to an end, we would greatly appreciate your feedback on whether our additional results and clarifications have adequately addressed your concerns. If you could kindly take a moment to review our updates and provide further guidance, we would be truly grateful. Your feedback will help us make more targeted and meaningful improvements to the final version. Thank you again for your time and insightful advice. We wish you all the best in your work!

Best, authors

AuthorsOpenReview ↗

Sincere reply to reviewer nN9g

Dear Reviewer nN9g,

We sincerely thank you again for your valuable comments and constructive suggestions.

In response to your concerns (Q1–Q2), we have conducted additional experiments and provided further analyses and clarifications in our rebuttal. We hope these updates have adequately addressed your concerns.

As the rebuttal period is approaching its deadline, we would be particularly grateful if you could kindly take a moment to review our responses and let us know whether any concerns remain. Your feedback would be extremely valuable in helping us further improve the paper and refine the camera-ready version.

We truly appreciate your time and thoughtful consideration, especially during this busy review period.

Best, authors

Reviewer_6tGYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MedScissor compresses projected medical-image tokens by ranking neighboring-token edges, merging groups under a size cap, and retaining one original token per group using centrality and global deviation. It is evaluated on six 2D and two 3D VQA benchmarks with Lingshu-7B and Hulu-Med-7B, respectively.

Critical strengths
  1. Practical interface: Compression uses projected features and grid coordinates without requiring internal attention maps.

  2. Competitive 2D results: Table 1 reports the highest mean at all three compression ratios, including 54.5 versus DivPrune's 54.1 at 50% compression.

  3. Useful evaluation coverage: Eight benchmarks, eight baselines, and depth-connectivity ablations examine both planar and volumetric settings.

Critical weaknesses
  1. The centrality ablation is degenerate at 50%. With γ=1\gamma=1, groups contain at most two tokens. Equation (8) assigns identical centrality to both members of every pair, so adding CiC_i cannot change selection for fixed groups. Table 4 therefore cannot establish a centrality benefit in this regime.
  2. Novelty is insufficiently differentiated. ST-SimDiff [1] already uses local spatiotemporal graphs and within-group similarity centrality. MedScissor adds neighborhood weighting, capped merging and prototype deviation, and avoids [1]'s attention-based budget correction. However, neither discussion nor a controlled comparison establishes the value of these differences.
  3. Medical evidence preservation remains indirect. Figure 1 motivates lesion-sensitive selection from one example; Figure 7 adds one qualitative case. Neither quantifies diagnostic-region retention across cases. Aggregate VQA retention does not establish that low-saliency lesions survive compression.
  4. Efficiency accounting is incomplete. Tables 6–9 report LLM prefill time without defining whether graph construction, sorting, rescanning and selection are included. End-to-end latency and explicit batch-size, precision and attention-backend settings are absent. These measurements do not establish end-to-end acceleration.

References:

[1] ST-SimDiff: Balancing Spatiotemporal Similarity and Difference for Efficient Video Understanding with MLLMs. In ICLR 2026.

Questions for authors
  1. What rounding or constraint-relaxation rule makes the 70% budget feasible, and what are the actual retained-token counts and group-size distributions?
  2. With grouping and decoding randomness fixed, do the full and w/o CiC_i variants select identical tokens at 50% compression? If not, which implementation detail explains this?
  3. At matched projected features and token budgets, what improvement remains over a graph-and-centrality baseline derived from [1]?
5 comments
AuthorsOpenReview ↗

Sincere reply to reviewer 6tGY (part1/2)

Dear Reviewer 6tGY,

Thank you for your valuable and insightful comments. We have consolidated the Weaknesses and Questions you raised into the following five points and address them individually below. We also commit to incorporating the relevant experimental results, implementation details, and clarifications into the camera-ready version.


Q1: On the effectiveness of local centrality at 50% compression (Weakness 1, Question 2)

A1: Thank you for this insightful observation. We agree that when a group contains only two tokens, their local centrality scores defined in Eq. (8) are identical and therefore cannot affect representative token selection. However, 50% compression does not necessarily imply that all groups contain at most two tokens.

Specifically, our implementation uses K=⌊N(1−r)⌋K=\lfloor N(1-r)\rfloor and Smax⁡=⌈γN/K⌉S_{\max}=\lceil\gamma N/K\rceil. The paper simplifies the latter as Smax⁡=γ/(1−r)S_{\max}=\gamma/(1-r), since the theoretical average group size satisfies N/K≈1/(1−r)N/K\approx1/(1-r). This continuous formulation simplifies the mathematical presentation, while the implementation further accounts for integer rounding.

When the original token count NN is odd, the group-size bound may allow groups containing three tokens. For example, for a 15×1515\times15 token grid, N=225N=225, K=112K=112, and Smax⁡=3S_{\max}=3. In such groups, local centrality scores are not necessarily identical and can therefore influence representative token selection.

Furthermore, the overall impact of local centrality is relatively small at 50% compression, but becomes more pronounced at higher compression ratios. As shown in Table 4, although the full method and the variant without local centrality exhibit performance differences on individual datasets, their average performance is identical at 50% compression (59.9 vs. 59.9). At 70% and 90% compression, the full method achieves higher average performance, with 59.1 vs. 58.9 and 56.9 vs. 56.4, respectively. We will clarify the integer rounding rules and the role of local centrality under different compression ratios in the camera-ready version.


Q2: On the novelty of MedScissor and its distinctions from ST-SimDiff (Weakness 2, Question 3)

A2: Thank you for pointing out the related work ST-SimDiff [1]. Although both methods employ local graph structures and within-group centrality, they differ in their motivations and compression mechanisms. ST-SimDiff primarily exploits spatiotemporal similarity and inter-frame differences to preserve important video events, and relies on LLM attention to adjust the final token budget. In contrast, MedScissor is specifically motivated by local structural variation and global visual deviation in medical images. It combines semantic similarity and neighborhood consistency for grouping, followed by progressive group merging and global deviation-aware representative selection, without requiring access to internal LLM attention.

To further validate these design choices, we conducted additional controlled experiments against ST-SimDiff [1]. Under identical projected visual features, token budgets, and evaluation protocols, we adopted its graph-based and centrality-based strategies, using spatial adjacency for 2D images and additional inter-slice connections for 3D volumes. We evaluated three 2D benchmarks and one 3D benchmark at three compression ratios, as summarized below.

Table R1. Performance comparison with ST-SimDiff.

CompressionMethodPMC-VQAPathVQAVQA-RADM3D-VQAAvg.
50%ST-SimDiff [1]60.254.552.681.062.08
MedScissor61.856.554.381.163.43
70%ST-SimDiff [1]59.254.353.080.461.73
MedScissor59.956.253.480.662.53
90%ST-SimDiff [1]55.653.150.678.359.40
MedScissor57.252.752.178.560.13

The results demonstrate that MedScissor consistently achieves higher average performance across all three compression ratios, outperforming ST-SimDiff by 1.35, 0.80, and 0.73 percentage points, respectively. These improvements support the effectiveness of our medical image-oriented grouping and representative selection strategies. In the camera-ready version, we will cite ST-SimDiff as relevant prior work, incorporate these additional experimental results, and further clarify the methodological distinctions between MedScissor and ST-SimDiff.

Reference

[1] ST-SimDiff: Balancing Spatiotemporal Similarity and Difference for Efficient Video Understanding with MLLMs. ICLR 2026.

AuthorsOpenReview ↗

Sincere reply to reviewer 6tGY (part2/2)

Q3: On the preservation of medical diagnostic evidence and lesion information (Weakness 3)

A3: Thank you for this valuable comment. We understand the reviewer's concern regarding the preservation of clinically relevant diagnostic evidence. Our evaluation covers eight medical imaging benchmarks, including PMC-VQA, PathVQA, SLAKE, VQA-RAD, MMMU-Med, MedXpertQA-MM, M3D-VQA, and 3D-RAD. These benchmarks involve radiological images, pathological images, and various diseases and clinical diagnostic scenarios, providing broad coverage of medical imaging tasks.

As shown in Tables 1 and 2, MedScissor consistently maintains high task performance across different compression ratios and outperforms existing compression methods on multiple benchmarks. These results indicate that our method can effectively preserve visual information important for medical understanding and diagnostic reasoning while substantially reducing visual tokens. Although aggregate VQA performance does not directly quantify lesion-region retention, the stable performance across diverse medical benchmarks provides strong indirect evidence of medical information preservation.

Furthermore, we will include additional visualization examples covering different medical imaging modalities and disease scenarios in the camera-ready version, further illustrating how MedScissor preserves lesion regions and other diagnostically relevant visual information.


Q4: On compression overhead and end-to-end inference efficiency (Weakness 4)

A4: Thank you for raising this important concern. We agree that explicitly clarifying the scope of computational overhead is necessary. First, the LLM Prefill Time reported in Tables 6–9 does not include the additional token compression overhead. The graph construction, edge sorting, repeated scanning, and representative token selection operations in MedScissor are lightweight, with an overall compression overhead of approximately 10–20 ms/sample. All experiments use Batch Size = 1, BF16 precision, and FlashAttention-2.

To further evaluate practical inference efficiency, we conducted additional end-to-end latency experiments that include the token compression overhead. The following results are averaged across all eight medical imaging benchmarks. Due to space constraints, we report comparisons with two competitive baselines, ToMe and LLaVA-PruMerge.

Table R2. End-to-End Latency Reduction (%), relative to the uncompressed model (↓ indicates lower latency).

Method50%70%90%
ToMe↓ 5.1%↓ 10.7%↓ 14.5%
LLaVA-PruMerge↓ 4.3%↓ 10.2%↓ 13.7%
MedScissor↓ 6.7%↓ 11.0%↓ 14.3%

The results demonstrate that MedScissor achieves meaningful end-to-end inference acceleration even after accounting for the additional token compression overhead. It outperforms both baselines at 50% and 70% compression while maintaining competitive efficiency at 90% compression. These findings further support the practical deployment value of MedScissor. We will include these results and clarify the experimental configurations and timing scope of each stage in the camera-ready version.


Q5: On token budget feasibility and group-size constraints at 70% compression (Question 1)

A5: Thank you for this detailed question. Consistent with our clarification in Q1, we determine the target number of retained tokens and the maximum group size using K=⌊N(1−r)⌋K=\lfloor N(1-r)\rfloor and Smax⁡=⌈γN/K⌉S_{\max}=\lceil\gamma N/K\rceil, respectively. The expression Smax⁡=γ/(1−r)S_{\max}=\gamma/(1-r) in the paper is a simplified continuous formulation that omits integer rounding effects.

Specifically, at 70% compression, the target number of retained tokens is K=⌊0.3N⌋K=\lfloor0.3N\rfloor. For example, when the original token count is N=1000N=1000, 300 tokens are retained, corresponding to an average group size of approximately 3.33 tokens. The resulting maximum group-size bound during constrained merging is 4 tokens. Therefore, MedScissor allows groups of varying sizes rather than enforcing uniform grouping, while limiting each group to at most four tokens during the constrained merging stage.

AuthorsOpenReview ↗

Sincere reply to reviewer 6tGY

Dear Reviewer 6tGY,

We would like to express our sincere appreciation for your valuable comments and insightful suggestions regarding our manuscript. In response to the issues and concerns you highlighted, we have carried out additional experiments and analyses (Q1–Q5), and we will incorporate the corresponding revisions and improvements into the camera-ready version. As the rebuttal period is approaching its conclusion, we would greatly appreciate it if you could take the time to review our updated responses and supplementary results, and let us know whether they have sufficiently addressed your concerns. Any further feedback or suggestions you may provide would be highly valuable and will help us refine the final version of our manuscript in a more targeted and effective manner. Once again, we sincerely thank you for your time, effort, and thoughtful feedback. We greatly appreciate your guidance and wish you continued success in your work.

Best, authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Appreciation to the Area Chair and Reviewers

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all reviewers for their time and effort in evaluating our paper. We are pleased that the reviewers recognized several strengths of our work, including comprehensive experimental evaluations with extensive comparisons and ablation studies (Reviewers 6tGY and HrFK), competitive performance and significant token compression effectiveness (Reviewers 6tGY, nN9g, and HrFK), practicality and ease of integration into existing models without additional training (Reviewers 6tGY and nN9g), and clear methodological illustrations and algorithmic descriptions (Reviewer HrFK).

We have carefully addressed the reviewers' comments and suggestions and incorporated additional experimental results and analyses into the updated PDF. We will further refine the presentation and experimental analysis in the camera-ready version to improve the completeness and clarity of our paper.

Finally, we would like to once again express our sincere gratitude to the AC and all reviewers for their valuable feedback, time, and effort throughout the review process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.