Adaptive Demonstration Ordering for In-Context Learning via Performance-Oriented Ordering Predictor

In-context learning (ICL) is highly sensitive to demonstration ordering: even with the same input query, retrieved demonstrations, and target LLM, different orderings can induce markedly different inference performance. However, existing demonstration ordering methods primarily rely on static proxy scores, such as similarity or entropy, which are often poorly aligned with downstream performance and ignore how ordering interacts with the input query, retrieved demonstrations, and target LLM. Therefore, we formulate demonstration ordering as a performance prediction task and propose Adaptive Demonstration Ordering for In-Context Learning (ADOICL), a performance-oriented and adaptive framework that selects demonstration orderings conditioned on the input query, the retrieved demonstrations, and the target LLM. Specifically, ADOICL learns an ordering predictor from offline performance supervision to estimate the downstream performance of candidate orderings and select the most promising ordering before a single target-LLM inference. We instantiate ADOICL in two forms: ADOICL-Seq, a sequence-level version that directly scores serialized query–ordering pairs, and ADOICL-Emb, an embedding-first version that embeds each query and retrieved demonstration once. ADOICL-Emb reuses these representations across candidate orderings, thereby avoiding repeated re-embedding and substantially reducing embedding time, so that the dominant embedding-time cost scales approximately linearly with the number of demonstrations. Empirically, ADOICL achieves state-of-the-art performance across 9 target LLMs, 5 tasks, and 8 datasets, covering classification, generation, and regression. It is plug-and-play with existing ICL pipelines, generalizes across LLMs, tasks, datasets, and different demonstration numbers, further supports visual ICL and few-shot training-data scenarios, and provides a clustering-based group-level scalability variant for many-shot settings.enhancer for existing ICL pipelines.