A new approach for detecting shifts in forecast accuracy

Working papers set out research in progress by our staff, with the aim of encouraging comments and debate.
Published on 13 April 2018

Staff Working Paper No. 721
By Ching-Wai (Jeremy) Chiu, Simon Hayes, George Kapetanios and Konstantinos Theodoridis

Forecasts play a critical role at inflation-targeting central banks, such as the Bank of England. Breaks in the forecast performance of a model can potentially incur important policy costs. Commonly used statistical procedures, however, implicitly put a lot of weight on type I errors (or false positives), which result in a relatively low power of tests to identify forecast breakdowns in small samples. We develop a procedure which aims at capturing the policy cost of missing a break. We use data-based rules to find the test size that optimally trades off the costs associated with false positives with those that can result from a break going undetected for too long. In so doing, we also explicitly study forecast errors as a multivariate system. The covariance between forecast errors for different series, though often overlooked in the forecasting literature, not only enables us to consider testing in a multivariate setting but also increases the test power. As a result, we can tailor the choice of the critical values for each series not only to the in-sample properties of each series but also to how the series for forecast errors covary.

PDFA new approach for detecting shifts in forecast accuracy

Was this page useful?
Add your details...