LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression

SVD-based low-rank compression has become a fast-growing direction for reducing the memory and computational cost of large language models (LLMs). However, meaningful comparison across existing studies remains difficult as prior evaluations use varied benchmarks, inconsistent ratios, and diverse setups, often failing to isolate low-rank effects from auxiliary techniques. As a result, it remains unclear whether reported gains reflect method-level improvements or differences in evaluation protocol. This lack of comparability highlights the need for a unified, reproducible evaluation platform. To address this problem, we present $\text{LowRankArena}$, a standardized evaluation platform for SVD-based LLM compression. LowRankArena unifies task versions, uniform-precision compression budgets, comparison regimes, and serving measurements, and provides a reproducible pipeline with over 3 TiB released compressed checkpoints. Using LowRankArena as a first standardized audit, we revisit representative publicly reproducible SVD methods under matched settings and observe that conclusions from isolated evaluations become more conditional once these fundamental evaluation assumptions are aligned: rankings change across backbones and keep ratios, multiple-choice accuracy can hide large perplexity degradation, and nominal low-rank savings yield workload-dependent and often limited end-to-end speedups. Our code is available at: https://anonymous.4open.science/r/lowrankarena-7883.