acceptodds
Under review as a conference paper at ICLR 2027

PipeSearch: Buying Composition Labels Only Where They Change the Pipeline

Abstract

Serving a large language model on a single accelerator asks quantization, pruning, low-rank factorization and cache compression to share one memory and throughput budget, so what is deployed is an ordered composition of methods rather than a single choice. Choosing that composition needs the pairwise interactions of the methods, yet they have to be bought: a pairwise label costs a full compression pass, reversing two methods yields a different model, and some pairings refuse rather than degrade. We propose PipeSearch, which buys a pairwise label only while it can still change the returned pipeline, that is, while the pipelines of its optimistic and pessimistic completions disagree. Decision-relevant acquisition observes the cheapest unobserved pair either completion applies, order-aware prefix search carries which method acted last and keeps a sequence apart from its reverse, and feasibility maintenance excludes a refused pair at expansion time. The key insight is that the observations which reveal the interaction structure are part of the decision rather than a preliminary to it. Across catalogues of four to twelve atoms, PipeSearch returns the pipeline of complete offline labelling while building 0.201 to 0.500 of the ordered pairs, at 75.1 accelerator-hours against 367.4 at . Paying for every candidate pair instead leaves perplexity, zero-shot average, peak memory and decode rate unchanged at both memory budgets, at 15.0 and 3.6 times the search cost. Under a cap of the fp16 footprint, where no single method fits, PipeSearch fits Llama-3.1-8B at a predicted 8.79 GB and 9.59 perplexity.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM compression pipelines, composed model compression, order-aware pipeline search, cost-aware label acquisition, memory-constrained LLM deployment, post-training quantization and pruning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_o88XOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes PipeSearch with three components: decision-relevant acquisition (buy the cheapest unobserved pair appearing in either the optimistic or pessimistic completion, only while the two disagree), an order-aware prefix search, and feasibility maintenance that excludes refused pairs at expansion time. On catalogues of 4 to 12 atoms, the authors report that the search returns the same pipeline as complete offline labelling while building 20-50% of the K(K-1) ordered pairs, and that the returned pipelines fit memory caps where no single method fits.

Critical strengths
  • The framing is reasonable: interaction labels between compression methods are expensive to obtain
  • The search design is cleanly specified.
Critical weaknesses
  • The paper is hard to read. There are too many self-coined terms, e.g. "pricing/buying". I think they should just use the plain meaning, e.g. the GPU cost of actually evaluating one composition.
  • As far as I understand, the data in the experiments is simulated, including the performance, latency and memory of the compressed models. Can the authors clarify this?
  • The additivity assumption (Eq. 10) is not validated, neither experimentally nor theoretically. It is exactly what turns the cost from K! pipelines into K(K-1) labels.
  • Same-pool compositions have structural semantics that the model cannot express. Example: low-rank followed by channel pruning - the low-rank reconstruction is full-size, so the pruning saving either vanishes or the composition is not well defined. In the paper this can only come out as a small drawn penalty or a random refusal; there is also no real example of which method pairs actually refuse. How to solve the structure issue?
Questions for authors

see Weaknesses

Reviewer_Ntf4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a method called PipeSearch, which tries to find the same compression pipeline that exhaustive testing would find, without testing every ordered pair.

From my understanding the method mainly does the following repeatedly:

  1. Finds the best pipeline under optimistic assumptions
  2. Finds one under pessimistic assumptions
  3. If they disagree, evaluates a pair that might resolve the disagreement (this is quite clever)
  4. Stops when both searches choose the same pipeline

The the paper essentially is trying to perform this contrastive search that are supposed to reduce the cost of what would otherwise be a quite expensive compression pipeline.

Critical strengths

To give a generous reading, the problem that paper argues does seem quite important. Compression techniques can interact with each other effective, and their order may matter. The proposed method pipesearch is author's best attempt to solve the problem. There is enough technical details that allow reader to reasonably reconstruct the method.

Critical weaknesses
  • The paper is really hard to read and follow, and to be very honest is quite badly written; i am unsure if it's mainly the wording or the structure but it very much could be both. Just to give some examples:
  1. What exactly does author meant by "atom" here? (first appear in the abstract and later in the main text). From the reading the atom seems to refer to a compression method at a specific setting/config. But why doesn't the author declares it or define it?
  2. What does author meant by "That structure has to be bought, one full compression pass and one evaluation sweep at a time." The term "bought" seems very out of place here.

Reading the introduction many times and i struggle to find what exact problem it is and how the proposed method are meant to address them

  • i am very concerned about the evaluation metrics used in the paper. To expand a little, “accelerator-hours” are assigned rather than timed on hardware. So the claimed 8.xx GB memory, 2675 tokens/s, 9.59 perplexity, and cost savings are model-generated predictions? Not actual real running results?

  • Indeed the paper mentioned every metric is produced by its composition model, and that no reported reading is measured on a served model.

  • Most gains concern search cost and pair interactions and refusals are largely synthetic experiments. This doesn't provide a convincing argument for having good empirical contribution.

Questions for authors
  1. Can you validate the selected pipelines by actually constructing and serving them?
  2. Where do the refusal rates and pairwise corrections come from? Do these distributions match real compression experiments?
  3. Why call configurations “atoms” instead of the clearer “method configurations” or something?
Reviewer_nqxeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the selection of an ordered pipeline of model-compression modules when the interaction between two modules is unknown and expensive to evaluate. A pairwise label records either an order-specific correction to predicted memory, throughput, and quality or a refusal indicating that the ordered pair cannot be composed. The proposed PipeSearch algorithm jointly chooses a deployment pipeline and the pairwise observations used to justify it. It repeatedly searches under an optimistic completion and a pessimistic completion of the unobserved pair table. When the two searches return different pipelines, it acquires the cheapest unobserved pair occurring in either candidate; when they agree, it builds and checks the agreed pipeline. The search state preserves order, while observed refusals are propagated during expansion.

The evaluation uses synthetic catalogues containing 4–12 atoms drawn from six compression families. Each atom has hand-specified or perturbed ratios for weight memory, KV-cache memory, throughput, and quality. Ordered-pair corrections and refusals are generated according to controlled distributions. Reported perplexity, zero-shot accuracy, memory, latency, batch size, and throughput are deterministic transformations of these simulated ratios rather than measurements from compressed and served models. Within this environment, PipeSearch matches the pipeline returned by complete offline labelling while acquiring 20.1%–50.0% of the ordered pairs. At catalogue size 12, its reported acquisition cost is 75.1 accelerator-hours versus 367.4 for complete labelling.

Critical strengths
  1. The paper exposes acquisition cost as part of the optimization problem.

Many composition-search formulations assume that a compatibility or interaction table is already available. Treating those observations as costly decisions is conceptually useful and could be consequential in settings where each composition requires a full model build and evaluation.

  1. The three mechanisms form a coherent search procedure.

Decision-relevant acquisition focuses measurements on the current candidates, the state representation preserves order, and refusal propagation prevents known-invalid regions from being repeatedly explored. The paper also provides detailed pseudocode, controlled ablations, and deterministic synthetic reproduction.

Critical weaknesses
  1. The empirical claims are not validated on real compressed or served models.

Every reported metric is produced by the composition model: atom effects come from parameter tables, pairwise corrections and refusals are generated, perplexity and task accuracy are obtained through fixed link functions, and accelerator-hours are accounting charges rather than measured runtime. Consequently, the Llama-3.1-8B and Qwen2.5-7B tables do not demonstrate that PipeSearch finds a feasible or high-quality pipeline for either backbone. This is central rather than a minor missing experiment because the paper's motivation depends on real order effects, failures, costs, and higher-order interactions.

  1. The decision rule lacks a sufficiently general correctness argument and relies on restrictive interaction structure.

Pipeline readouts are modeled as products of solo effects plus additive ordered-pair corrections. Real compositions may contain higher-order and state-dependent interactions. In addition, the “optimistic” completion maximizes a fixed scalar utility over corrections and is explicitly not componentwise optimistic. It is therefore unclear why agreement between that completion and the all-refusal completion should certify that additional labels cannot change the returned constrained optimum. The reported identity to the complete-label oracle is empirical and restricted to the same small synthetic generator, with at most 12 atoms.

Questions for authors
  1. Can the authors state and prove conditions under which agreement between the selected optimistic and pessimistic completions guarantees that every completion consistent with the acquired labels returns the same optimal pipeline? If no such guarantee is intended, how often does the agreement stop return a different pipeline from complete labelling under broader, adversarial, or out-of-distribution pair tables?

  2. Can the authors evaluate PipeSearch using actual implementations of at least a small set of compression families, measuring both orders of selected pairs and the final multi-method pipelines on real models and hardware? This would materially affect my assessment because all current quality, feasibility, throughput, and acquisition-cost results are generated rather than observed.

  3. How were the solo-effect table, refusal densities, pairwise-correction distributions, order-gap levels, and build-cost charges calibrated? In particular, what empirical evidence supports the pairwise-additive model and the assumption that almost all non-refusal corrections are penalties?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.