School Brown Bag: Patrick Marsh

Speaker:  Patrick Marsh

Title: Self-Correction for interval and probability forecasts, with application to UK inflation.

Abstract:  Suppose that we wish to create interval and/or probability forecasts for some variable of interest, such as UK inflation. We specify and estimate a candidate forecast model which can then be evaluated either via a (proper) score function evaluated at the subsequent realisations or via a test for uniformity of the Out of Sample Probability Integral Transforms (PIT), i.e. the fitted distribution function also evaluated at the subsequent realisations. However, it is not clear how to proceed in the event the latter is rejected nor how any of the inevitable forecast score loss can be recovered.

This work creates a self-correction for out of sample forecasts, based on nonparametric estimation of the density of the in-sample PITs. It is proved that the correction improves the forecast as measured by the log-score. In some circumstances, such as one-step ahead forecasts based on location-scale type time series, the entirety of the forecast loss can be recovered. Moreover, in any circumstance including longer forecast horizons, some of the loss can be recovered by the correction. Monte Carlo experiments demonstrate the loss recovery, both in terms of the score function itself but also in terms of the width and coverage of forecast intervals. There is also evidence that the correction will recover some of the loss implied by parameter uncertainty, i.e. having to estimate unknown model parameters. Even notionally correctly specified forecast models can be corrected.

For UK RPI inflation, from 1980 to present, the correction is applied to both a simple (and atheoretical) Gaussian integrated moving average as well as a backwards looking Phillips curve. The corrected forecast intervals have significantly improved coverage and are far narrower, up to 20% better, for both.