- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes PipeSearch with three components: decision-relevant acquisition (buy the cheapest unobserved pair appearing in either the optimistic or pessimistic completion, only while the two disagree), an order-aware prefix search, and feasibility maintenance that excludes refused pairs at expansion time. On catalogues of 4 to 12 atoms, the authors report that the search returns the same pipeline as complete offline labelling while building 20-50% of the K(K-1) ordered pairs, and that the returned pipelines fit memory caps where no single method fits.
Critical strengths
- The framing is reasonable: interaction labels between compression methods are expensive to obtain
- The search design is cleanly specified.
Critical weaknesses
- The paper is hard to read. There are too many self-coined terms, e.g. "pricing/buying". I think they should just use the plain meaning, e.g. the GPU cost of actually evaluating one composition.
- As far as I understand, the data in the experiments is simulated, including the performance, latency and memory of the compressed models. Can the authors clarify this?
- The additivity assumption (Eq. 10) is not validated, neither experimentally nor theoretically. It is exactly what turns the cost from K! pipelines into K(K-1) labels.
- Same-pool compositions have structural semantics that the model cannot express. Example: low-rank followed by channel pruning - the low-rank reconstruction is full-size, so the pruning saving either vanishes or the composition is not well defined. In the paper this can only come out as a small drawn penalty or a random refusal; there is also no real example of which method pairs actually refuse. How to solve the structure issue?
Questions for authors
see Weaknesses