Staff Working Paper No. 1,190
Nikoleta Anesti, Edward Hill and Andreas Joseph
This paper investigates the ability of large language models (LLMs), primarily ‘GPT-3.5 Turbo’ (GPT), to form inflation perceptions and expectations based on macroeconomic price signals. We compare the LLM’s output to household survey data and official statistics, mimicking the information set and demographic characteristics of the Bank of England’s Inflation Attitudes Survey (IAS). Our quasi-experimental design exploits the timing of GPT’s training cut-off in September 2021 which means it has no knowledge of the subsequent UK inflation surge. This setting turns out to be crucial to track aggregate survey results and official statistics at short horizons. At a disaggregated level, GPT replicates key empirical regularities of households’ inflation perceptions, particularly for income, housing tenure, and social class. A novel Shapley value decomposition of LLM outputs suited for the synthetic survey setting provides well-defined insights into the drivers of model outputs linked to prompt content. We find that GPT demonstrates a heightened sensitivity to food inflation information like that of human respondents. However, we also find that it lacks a consistent model of consumer price inflation, eg by exhibiting unexplained kinks in component sensitivity. More generally, our approach could be used to evaluate the behaviour of LLMs for use in the social sciences, to compare different models, or to assist in survey design.
Inflation attitudes of large language models