Temporal Concepts and their Shape in Large Language Models
Desk rejected at ICLR 2027, so left out of the paper lists and the map.
Abstract
Large Language Models (LLMs) are increasingly being deployed to make decisions that require trading off near-term gains against long-term consequences, yet little is known about how they represent these tradeoffs internally. We localize temporal concepts to their fractional depth, across families (Qwen, Llama, Gemma, Mistral), scales (Qwen 0.6B to 14B), and domains (financial, health, climate, education, entrepreneurial). Around those fractional depths, time horizon forms an ordinal gradient. These concepts undergo complex transformations across token positions. We measure temporal preference stability and temporal reasoning coherence. Every model shows temporal structure, but the representational manifold is not always functional, as it does not imply good intertemporal behavior. Our work advocates for greater oversight of LLM temporal behavior.
- Primary area:
- interpretability and explainable AI