acceptodds
Under review as a conference paper at ICLR 2027

LoRA-Ring: Locality-Preserving Consistent Hashing for Dynamic Adapter Routing in Large Language Models

Abstract

Serving large language models with many LoRA adapters requires routing each request to a server that has the relevant adapter loaded, since adapter loading latency (200–800ms) dominates request latency when adapters are not cached. Existing routers assign adapters to servers using static hash partitioning, which ignores adapter co-access locality—pairs of adapters that tend to appear in consecutive requests from the same user. LoRA-Ring introduces locality-preserving consistent hashing that places co-accessed adapters on adjacent ring segments, enabling a single server to serve common adapter pairs without cross-server fetches. We prove that LoRA-Ring reduces cross-server fetches by a factor proportional to the locality score of the workload, and show empirically that it reduces median request latency by 31% on a trace from a production multi-tenant serving system.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
LoRA, adapter routing, consistent hashing, LLM serving, multi-tenant inference, locality-preserving hashing

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_owizOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents a framework for dynamic LoRA adapter routing based on consistent hashing. The main motivation is to support adapter insertion and removal without retraining a routing model. Each adapter is represented by the semantic centroid of its training corpus. Incoming queries are routed using a clockwise successor lookup. The framework further introduces virtual nodes for load balancing and weighting for multi-adapter blending. No empirical measurements are reported.

Critical strengths
  1. The paper discusses a problem in maintaining large and dynamically evolving adapter libraries.
  2. The presentation is generally clear.
Critical weaknesses
  1. The theoretical contributions are from classical consistent hashing. However, the proposed method differs from the standard setting because adapter positions are determined by semantic centroids rather than independent, uniformly distributed hash values. The theoretical guarantees are not sufficiently justified.

  2. The method requires semantically similar adapters to occupy nearby ring positions, while the load-balancing guarantees of traditional consistent hashing rely on sufficiently dispersed virtual-node placements. These requirements may conflict.

  3. The paper does not sufficiently distinguish the advantages from those of a simple nearest-centroid router. The advantages remain unclear.

  4. Although the work is positioned as purely theoretical, it does not establish substantial new theorems beyond existing consistent-hashing results, and the applicability of these results to the proposed method remains insufficiently justified. No experiments are presented.

Questions for authors

See weaknesses.

Reviewer_eCEUOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes LoRA-Ring, a theoretical framework for dynamic LoRA adapter routing based on locality-preserving consistent hashing. Each adapter is represented by the semantic centroid of its training corpus and projected onto a one-dimensional hash ring. Incoming queries are embedded into the same space and routed using clockwise successor lookup.

The authors claim that this construction enables efficient lookup, bounded routing disruption under adapter additions and removals, and load-balancing properties inherited from consistent hashing. The paper also introduces distance-based adapter blending and discusses potential caching benefits for multi-adapter LLM serving.

Critical strengths
  1. The research problem is practically relevant. Supporting dynamic adapter additions and removals while maintaining routing stability is an important problem in multi-LoRA adapter serving systems.
  2. The proposed framework is relatively simple. By combining semantic centroids with consistent hashing, the approach avoids training a separate router and leverages consistent-hashing properties to analyze routing stability.
  3. The paper acknowledges several limitations. The authors discuss unresolved issues such as projection distortion, centroid stability, and task-level routing accuracy, and outline possible directions for future validation.
Critical weaknesses

Weakness 1: The effectiveness of semantic adapter routing lacks theoretical or empirical support

The theoretical analysis of this paper mainly focuses on the lookup complexity, routing stability, and load balancing of consistent hashing, without establishing or empirically validating the effectiveness of adapter selection. First, the paper does not provide a concrete locality-preserving projection function or a distortion bound, making it unclear whether semantic neighborhood relationships in the original high-dimensional embedding space can be preserved on the one-dimensional hash ring. Second, clockwise successor lookup is not equivalent to nearest-neighbor search. Even if the projection preserves some distance relationships, it does not guarantee that the selected adapter is the most appropriate one in the original semantic space.

Weakness 2: The independent theoretical contribution and practical systems benefits are insufficiently supported

Most of the paper's theoretical properties are inherited from classical consistent hashing. In particular, Section 12 categorizes the relevant claims as existing theorems, direct corollaries, or open questions, leaving the paper's novel theoretical contributions unclear.

Furthermore, the claimed systems benefits are insufficiently justified. The adapter-level balancing of query counts in Definition 4 does not necessarily translate into balanced GPU workloads, which also depend on adapter placement, request scheduling, and workload characteristics. The paper does not sufficiently establish a connection between its load-balancing properties and actual inference-serving performance.

Questions for authors
  1. Routing accuracy: What concrete locality-preserving projection function do the authors intend to use? Can the authors provide a distortion bound and analyze the relationship between clockwise successor routing and nearest-centroid routing in the original semantic embedding space? Can this routing mechanism guarantee effective adapter selection in practice?
  2. Load balancing: Why is balancing the number of queries assigned to each adapter an appropriate optimization objective? How can this objective be reconciled with semantic routing accuracy, and how does it translate into improvements in GPU utilization, throughput, or latency?
  3. Expert blending: Section 6 selects the top-k adapters based on ring distance, but geometric proximity does not necessarily imply complementary expertise. For queries requiring knowledge from multiple domains, how does the proposed mechanism ensure that relevant adapters are not excluded simply because they are distant on the hash ring?
  4. Experimental validation: The paper currently lacks experimental evaluation. I encourage the authors to provide experiments evaluating the actual adapter-selection accuracy of LoRA-Ring, routing disruption under dynamic adapter additions and removals, load balancing, and inference-serving performance benefits, with comparisons against appropriate existing routing approaches.
Reviewer_b4njOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes LoRA-Ring, a purely theoretical framework that treats LoRA adapter routing as a consistent-hashing problem. Each adapter is represented by the centroid of its training-corpus embeddings and projected onto a 1D ring via a locality-preserving map with virtual nodes. Each query is routed to its clockwise successor on the ring. The paper claims to inherit monotonicity under churn, load balance, and O(log(nv)) lookup from consistent hashing. It also adds soft blending by ring distance and a qualitative cache-locality argument. There are no experiments and no new proofs.

Critical strengths

The framing of adapter routing as a coordination problem, with churn stability as an explicit design goal separate from accuracy, is a reasonable perspective. The paper is also unusually transparent about which of its claims are conditional or untested. Neither point is enough to outweigh the weaknesses below.

Critical weaknesses
  1. The ring adds nothing over exact nearest-centroid routing, and it costs accuracy.
  2. The inherited guarantees do not transfer to this setting.

With no experiments and no new theory, these issues leave the contribution unsupported.

Questions for authors
  1. What concrete property does LoRA-Ring provide that exact or ANN nearest-centroid routing in Rd\mathbb{R}^d does not?
  2. Why use the clockwise successor rather than the nearest ring neighbor in either direction, given Def. 2?
  3. Under semantic placement and correlated virtual nodes, what bounds actually hold in place of Props. 1 and 2?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.