MMSE Forecasting Regression Models with ARIMA Errors

What Are MMSE Forecasts?

An objective of time series analysis is generating forecasts for responses over a future time horizon. That is, you can generate predictions for y_{T + 1},y_{T + 2},...,y_{T + h} given the following:

• An observed series y₁, y₂,...,y_T

• A forecast horizon h

• Nonstochastic predictors x₁,x₂,...,x_T,...,x_{T + h}, where x_k is an r-vector containing the measurements of r predictors observed at time k

• A regression model with ARIMA errors

  $$\begin{array}{l}{y}_t=c+X_t\beta +u_t\\ {\rm H}(L)u_t={\rm N}(L){\epsilon }_t,\end{array}$$

  where H(L) and N(L) are compound autoregressive and moving average lag operator polynomials (possibly containing integration), respectively.

Let $${\widehat{y}}_{t+1}$$ denote a forecast for the process at time t + 1, conditional on the history of the process up to time t (H_t), and assume that the predictors are fixed. The minimum mean square error (MMSE) forecast is the forecast $${\widehat{y}}_{t+1}$$ that minimizes expected square loss,

$$E{\left(y_{t+1}-{\widehat{y}}_{t+1}|H_t\right)}^2.$$

Minimizing this loss function yields the MMSE forecast,

$${\widehat{y}}_{t+1}=E(y_{t+1}|H_t).$$

How forecast Generates MMSE Forecasts

forecast generates MMSE forecasts recursively. When you call forecast, you must specify a regARIMA model (Mdl) and the forecast horizon. You can also specify presample observations (Y0), predictors (X0), innovations (E0), and conditional disturbances (U0) using name-value pair arguments.

To begin forecasting y_t starting at time T + 1, use the last few observations of y_t and X_t as presample responses and predictors to initialize the forecast. Alternatively, you can specify presample unconditional disturbances or innovations.

However, when you specify presample data:

• If you provide presample predictor data (X0), then you must also provide predictor forecasts (XF). It is best practice to set X0 to the same predictor matrix that estimates the parameters. If you do not provide presample and future predictors, then forecast ignores the regression component in the model.

• If the error process in Mdl contains a seasonal or nonseasonal autoregressive component, or seasonal or nonseasonal integration, then forecast requires a minimum of P presample unconditional disturbances to initialize the forecast. The property P of Mdl stores P.

• If the error process in Mdl contains a seasonal or nonseasonal moving average component, then forecast requires a minimum of Q presample innovations to initialize the forecast. The property Q of Mdl stores Q.

• If you provide a sufficient amount of presample unconditional disturbances, then forecast ignores Y0 and X0. If you also do not provide E0, but provide enough presample unconditional disturbances, then forecast infers the required amount of presample innovations from the ARIMA error model and U0.

• If you provide a sufficient amount of presample responses and predictors (and do not provide U0), then forecast uses the regression model to infer the presample unconditional disturbances.

• If you do not provide presample observations, then forecast sets the required amount of presample unconditional disturbances and innovations to 0.

• If you provide an insufficient amount of presample observations, then forecast returns an error.

Consider generating forecasts from a regression model with ARMA(3,2) errors:

$$\begin{array}{c}{y}_t=c+X_t\beta +u_t\\ \left(1-a_{1}L-a_2{L}^2-a_3{L}^3\right)u_t=\left(1+b_{1}L+b_2{L}^2\right){\epsilon }_t\\ \text{or}\\ a(L)u_t=b(L){\epsilon }_t,\end{array}$$

where a(L) and B(L) are lag operator polynomials. The largest AR lag is 3, the largest MA lag is 2. This model does not contain any seasonal lags nor integration. Therefore, P = 3 and Q = 2. To forecast this model, you need three presample responses and predictors, or three presample unconditional disturbances, and two presample innovations.

Given presample unconditional disturbances $$\left(u_{T-2},u_{T-1},u_T\right),$$ presample innovations $$\left({\epsilon }_{T-1},{\epsilon }_T\right),$$ and future predictors $$\left(X_{T+1},X_{T+2},\mathrm{...}\right),$$ you can forecast the model as follows:

• $$\begin{array}{l}{\widehat{u}}_{T+1}=a_1{u}_T+a_2{u}_{T-1}+a_3{u}_{T-2}+b_1{\epsilon }_T+b_2{\epsilon }_{T-1}\\ {\widehat{y}}_{T+1}=c+X_{T+1}\beta +{\widehat{u}}_{T+1}.\end{array}$$

• $$\begin{array}{l}{\widehat{u}}_{T+2}=a_1{\widehat{u}}_{T+1}+a_2{u}_T+a_3{u}_{T-1}+b_2{\epsilon }_T\\ {\widehat{y}}_{T+2}=c+X_{T+2}\beta +{\widehat{u}}_{T+2}.\end{array}$$

• $$\begin{array}{l}{\widehat{u}}_{T+3}=a_1{\widehat{u}}_{T+2}+a_2{\widehat{u}}_{T+1}+a_3{u}_T\\ {\widehat{y}}_{T+3}=c+X_{T+3}\beta +{\widehat{u}}_{T+3}.\end{array}$$

...

Note that:

• Future innovations take on their unconditional mean, 0.

• For stationary error processes, such as this one:

  • The forecasted unconditional disturbances converge to their unconditional mean,

    $$E(u_t)=\frac{b(L)}{a(L)}E({\epsilon }_t)=0.$$

  • c + X_tβ governs the long-term behavior of the forecasted responses.

Forecast Error

The forecast error for an s-step ahead forecast of a regression model with ARIMA errors is

$$\begin{array}{c}\text{MSE}=E{\left(y_{T+s}-{\widehat{y}}_{T+s}|H_{T+s-1}\right)}^2\\ =E{\left(c+X_{T+s}\beta +u_{T+s}-c-X_{t+s}\beta -{\widehat{u}}_{T+s}|H_{T+s-1}\right)}^2\\ =E{\left(u_{T+s}-{\widehat{u}}_{T+s}|H_{T+s-1}\right)}^2\\ =\frac{{\rm N}(L)}{{\rm H}(L)}E\left({\epsilon }_t{}^2|H_{T+s-1}\right)\\ =\psi (L){\sigma }^2,\end{array}$$

where the dividend ψ(L) is an infinite lag operator polynomial, and σ² is the innovation variance.

If the error process is stationary, then the coefficients of ψ(L) are absolutely summable. Therefore, the MSE (mean square error) converges to the unconditional variance of the process [1].

If the error process is not stationary, then the MSE grows with increasing s.

References

[1] Box, G. E. P., G. M. Jenkins, and G. C. Reinsel. Time Series Analysis: Forecasting and Control. 3rd ed. Englewood Cliffs, NJ: Prentice Hall, 1994.