Proximity, Precendence and Position: A Minimum Linear Arrangement Account of Ordering Effects in LLM Prompts

In-context learning is highly sensitive to the ordering of demonstrations, but the root factors of context ordering remain undiscovered. Existing work treats a demonstration set as an unordered bag and searches over permutations based on a certain metric, without modeling the underlying relational structure. We give a structural account of order sensitivity built on three key properties of an arrangement: proximity (how far apart dependent facts sit), precedence (whether each fact appears after the facts it depends on), and position (where a fact sits in the context). We model proximity and precedence using minimum linear arrangement (MinLA), with precedence as a hard constraint and isolate position through controlled experiments. We treat the MinLA-constrained order as an oracle and measure how much of the total order effect it recovers relative to the worst achievable order. This provides a direct measure of how well our account explains observed behavior. On small-scale instances with known ground-truth ordering accuracy, it recovers 81% of the order effect in the referential domain and 63% in the associative domain. Across two domains with different dependency semantics — referential (GSM8K arithmetic, where a step reuses an unresolved prior value) and associative (synthetic narratives, where a sentence names a concept introduced earlier) — orders that are both MinLA-optimal and precedence-legal outperform the natural order by +34.5% accuracy in arithmetic and +16.3% in narrative on Llama-3.2-3B-Instruct, while MinLA optimization without the legality constraint does not reliably help. Decomposing the order effect into clustering (proximity) and direction (precedence), direction accounts for 61% of it in arithmetic and clustering for 56% in narrative, consistent with a referential-versus-associative account of how each domain encodes dependency. A third, independent, U-shaped position effect, isolated under experimental control, connects our results to lost-in-the-middle phenomena reported in previous work only observationally.