Distant Memories: How do LLMs remember financial data?

The nature of information storage and access in large language models is far from fully understood. We focus on knowledge of financial information, and to this end investigate the depth and nature of LLMs' knowledge of prices of instruments such as stocks, currencies and indices. We also investigate how this varies with the size and type of model, and whether it is affected by how the information is recalled. To yield a baseline, we investigate the retrieval of non-existent instrument prices, including those that are similar to existing instruments. We show that LLMs store data in different ways depending on the frequency of it appearing in the training data; we also show that the retrieval process is a two-way process where certain information is more helpful in retrieval than others.