Create Regression Models with ARIMA Errors

This topic shows how to represent various regression models with autoregressive integrated moving average (ARIMA) errors, which are a type of univariate time series regression model, as a regARIMA model object. Also, the topic shows how to interpret the property values of a specified object.

Default Regression Model with ARIMA Errors Specifications

Regression models with ARIMA errors have the following form (in lag operator notation):

$$\begin{array}{c}{y}_t=c+X_t\beta +u_t\\ a\left(L\right)A\left(L\right){\left(1-L\right)}^D\left(1-L^s\right)u_t=b\left(L\right)B\left(L\right){\epsilon }_t,\end{array}$$

where

• t = 1,...,T.

• y_t is the response series.

• X_t is row t of X, which is the matrix of concatenated predictor data vectors. That is, X_t is observation t of each predictor series.

• c is the regression model intercept.

• β is the regression coefficient.

• u_t is the disturbance series.

• ε_t is the innovations series.

• $$L^j{y}_t=y_{t-j}.$$

• $$a\left(L\right)=\left(1-a_{1}L-\mathrm{...}-a_p{L}^p\right),$$ which is the degree p, nonseasonal autoregressive polynomial.

• $$A\left(L\right)=\left(1-A_{1}L-\mathrm{...}-A_{p_s}{L}^{p_s}\right),$$ which is the degree p_s, seasonal autoregressive polynomial.

• $${\left(1-L\right)}^D,$$ which is the degree D, nonseasonal integration polynomial.

• $$\left(1-L^s\right),$$ which is the degree s, seasonal integration polynomial.

• $$b\left(L\right)=\left(1+b_{1}L+\mathrm{...}+b_q{L}^q\right),$$ which is the degree q, nonseasonal moving average polynomial.

• $$B\left(L\right)=\left(1+B_{1}L+\mathrm{...}+B_{q_s}{L}^{q_s}\right),$$ which is the degree q_s, seasonal moving average polynomial.

For simplicity, use the shorthand notation Mdl = regARIMA(p,D,q) to specify a regression model with ARIMA(p,D,q) errors, where p, D, and q are nonnegative integers. Mdl has the following default properties.

Property Name	Property Data Type
AR	Length p cell vector of NaNs
Beta	Empty vector [] of regression coefficients, corresponding to the predictor series
D	Nonnegative scalar, corresponding to D
Distribution	"Gaussian", corresponding to the distribution of εt
Intercept	NaN, corresponding to c
MA	Length q cell vector of NaNs
P	Number of AR terms plus degree of integration, p + D
Q	Number of MA terms, q
SAR	Empty cell vector
SMA	Empty cell vector
Variance	NaN, corresponding to the variance of εt
Seasonality	0, corresponding to s

If you specify nonseasonal ARIMA errors, then

• The properties D and Q are the inputs D and q, respectively.

• Property P = p + D, which is the degree of the compound, nonseasonal autoregressive polynomial. In other words, P is the degree of the product of the nonseasonal autoregressive polynomial, a(L) and the nonseasonal integration polynomial, (1 – L)^D.

The values of properties P and Q indicate how many presample observations the software requires to initialize the time series.

You can modify the properties of Mdl using dot notation. For example, Mdl.Variance = 0.5 sets the innovation variance to 0.5.

For maximum flexibility in specifying a regression model with ARIMA errors, use name-value pair arguments to, for example, set each of the autoregressive parameters to a value, or specify multiplicative seasonal terms. For example, Mdl = regARIMA('AR',{0.2 0.1}) defines a regression model with AR(2) errors, and the coefficients are a₁ = 0.2 and a₂ = 0.1.

Specify regARIMA Models Using Name-Value Pair Arguments

You can only specify the nonseasonal autoregressive and moving average polynomial degrees, and nonseasonal integration degree using the shorthand notation regARIMA(p,D,q). Some tasks, such as forecasting and simulation, require you to specify values for parameters. You cannot specify parameter values using shorthand notation. For maximum flexibility, use name-value pair arguments to specify regression models with ARIMA errors.

The nonseasonal ARIMA error model might contain the following polynomials:

• The degree p autoregressive polynomial a(L) = 1 – a₁L – a₂L₂ –...– a_pL^p. The eigenvalues of a(L) must lie within the unit circle (i.e., a(L) must be a stable polynomial).

• The degree q moving average polynomial b(L) = 1 + b₁L + b₂L₂ +...+ b_qL^q. The eigenvalues of b(L) must lie within the unit circle (i.e., b(L) must be an invertible polynomial).

• The degree D nonseasonal integration polynomial is (1 – L)^D.

The following table contains the name-value pair arguments that you use to specify the ARIMA error model (i.e., a regression model with ARIMA errors, but without a regression component and intercept):

$$\begin{array}{c}{y}_t=u_t\\ a(L){(}^1=b(L){\epsilon }_t.\end{array}$$

(1)

Name-Value Pair Arguments for Nonseasonal ARIMA Error Models

Name	Corresponding Model Term(s) in Equation 1	When to Specify
AR	Nonseasonal AR coefficients: a1, a2,...,ap	To set equality constraints for the AR coefficients. For example, to specify the AR coefficients in the ARIMA error model $$u_t=0.8u_{t-1}-0.2u_{t-2}+{\epsilon }_t,$$ specify 'AR',{0.8,-0.2}. You only need to specify the nonzero elements of AR. If the nonzero coefficients are at nonconsecutive lags, specify the corresponding lags using ARLags. The coefficients must correspond to a stable AR polynomial.
ARLags	Lags corresponding to nonzero, nonseasonal AR coefficients	ARLags is not a model property. Use this argument as a shortcut for specifying AR when the nonzero AR coefficients correspond to nonconsecutive lags. For example, to specify nonzero AR coefficients at lags 1 and 12, e.g., $$u_t=a_1{u}_{t-1}+a_2{u}_{t-12}+{\epsilon }_t,$$ specify 'ARLags',[1,12]. Use AR and ARLags together to specify known nonzero AR coefficients at nonconsecutive lags. For example, if in the given AR(12) error model with a1 = 0.6 and a12 = –0.3, then specify 'AR',{0.6,-0.3},'ARLags',[1,12].
D	Degree of nonseasonal differencing, D	To specify a degree of nonseasonal differencing greater than zero. For example, to specify one degree of differencing, specify 'D',1. By default, D has value 0 (meaning no nonseasonal integration).
Distribution	Distribution of the innovation process, εt	Use this argument to specify a Student’s t distribution. By default, the innovation distribution is "Gaussian". For example, to specify a t distribution with unknown degrees of freedom, specify 'Distribution','t'. To specify a t innovation distribution with known degrees of freedom, assign Distribution a structure with fields Name and DoF. For example, for a t distribution with nine degrees of freedom, specify 'Distribution',struct('Name','t','DoF',9).
MA	Nonseasonal MA coefficients: b1, b2,...,bq	To set equality constraints for the MA coefficients. For example, to specify the MA coefficients in the ARIMA error model $$u_t={\epsilon }_t+0.5{\epsilon }_{t-1}+0.2{\epsilon }_{t-2},$$ specify 'MA',{0.5,0.2}. You only need to specify the nonzero elements of MA. If the nonzero coefficients are at nonconsecutive lags, specify the corresponding lags using MALags. The coefficients must correspond to an invertible MA polynomial.
MALags	Lags corresponding to nonzero, nonseasonal MA coefficients	MALags is not a model property. Use this argument as a shortcut for specifying MA when the nonzero MA coefficients correspond to nonconsecutive lags. For example, to specify nonzero MA coefficients at lags 1 and 4, e.g., $$u_t={\epsilon }_t+b_1{\epsilon }_{t-1}+b_4{\epsilon }_{t-4},$$ specify 'MALags',[1,4]. Use MA and MALags together to specify known nonzero MA coefficients at nonconsecutive lags. For example, if in the given MA(4) error model b1 = 0.5 and b4 = 0.2, specify 'MA',{0.4,0.2},'MALags',[1,4].
Variance	Scalar variance, σ2, of the innovation process, εt	To set equality constraints for σ2. For example, for an ARIMA error model with known innovation variance 0.1, specify 'Variance',0.1. By default, Variance has value NaN.

Use the name-value pair arguments in the following table in conjunction with those in Name-Value Pair Arguments for Nonseasonal ARIMA Error Models to specify the regression components of the regression model with ARIMA errors:

$$\begin{array}{c}{y}_t=c+X_t\beta +u_t\\ a(L){(}^1=b(L){\epsilon }_t.\end{array}$$

(2)

Name-Value Pair Arguments for the Regression Component of the regARIMA Model

Name	Corresponding Model Term(s) in Equation 2	When to Specify
Beta	Regression coefficient values corresponding to the predictor series, β	Use this argument to specify the values of the coefficients of the predictor series. For example, use 'Beta',[0.5 7 -2] to specify $$\beta ={\left[\begin{array}{ccc}0.5& 7& -2\end{array}\right]}^{\prime }.$$ By default, Beta is an empty vector, [].
Intercept	Intercept term for the regression model, c	To set equality constraints for c. For example, for a model with no intercept term, specify 'Intercept',0. By default, Intercept has value NaN.

If the time series has seasonality s, then

• The degree p_s seasonal autoregressive polynomial is A(L) = 1 – A ₁L – A₂L² –...– A_psL^p_s.

• The degree q_s seasonal moving average polynomial is B(L) 1 + B ₁L + B₂L² +...+ B_qsL^q_s.

• The degree s seasonal integration polynomial is (1 – L^s).

Use the name-value pair arguments in the following table in conjunction with those in tables Name-Value Pair Arguments for Nonseasonal ARIMA Error Models and Name-Value Pair Arguments for the Regression Component of the regARIMA Model to specify the regression model with multiplicative seasonal ARIMA errors:

$$\begin{array}{c}{y}_t=c+X_t\beta +u_t\\ a(L){(}^{1}A(L)(1-L^s)u_t=b(L)B(L){\epsilon }_t.\end{array}$$

(3)

Name-Value Pair Arguments for Seasonal ARIMA Models

Argument	Corresponding Model Term(s) in Equation 3	When to Specify
SAR	Seasonal AR coefficients: A1, A2,...,Aps	To set equality constraints for the seasonal AR coefficients. Use SARLags to specify the lags of the nonzero seasonal AR coefficients. Specify the lags associated with the seasonal polynomials in the periodicity of the observed data (e.g., 4, 8,... for quarterly data, or 12, 24,... for monthly data), and not as multiples of the seasonality (e.g., 1, 2,...). For example, to specify the ARIMA error model $$(1-0.8L)(1-0.2L^{12})u_t={\epsilon }_t,$$ specify 'AR',0.8,'SAR',0.2,'SARLags',12. The coefficients must correspond to a stable seasonal AR polynomial.
SARLags	Lags corresponding to nonzero seasonal AR coefficients, in the periodicity of the responses	SARLags is not a model property. Use this argument when specifying SAR to indicate the lags of the nonzero seasonal AR coefficients. For example, to specify the ARIMA error model $$(1-a_{1}L)(1-A_{12}{L}^{12})u_t={\epsilon }_t,$$ specify 'ARLags',1,'SARLags',12.
SMA	Seasonal MA coefficients: B1, B2,...,Bqs	To set equality constraints for the seasonal MA coefficients. Use SMALags to specify the lags of the nonzero seasonal MA coefficients. Specify the lags associated with the seasonal polynomials in the periodicity of the observed data (e.g., 4, 8,... for quarterly data, or 12, 24,... for monthly data), and not as multiples of the seasonality (e.g., 1, 2,...). For example, to specify the ARIMA error model $$u_t=(1+0.6L)(1+0.2L^4){\epsilon }_t,$$ specify 'MA',0.6,'SMA',0.2,'SMALags',4. The coefficients must correspond to an invertible seasonal MA polynomial.
SMALags	Lags corresponding to the nonzero seasonal MA coefficients, in the periodicity of the responses	SMALags is not a model property. Use this argument when specifying SMA to indicate the lags of the nonzero seasonal MA coefficients. For example, to specify the model $$u_t=(1+b_{1}L)(1+B_4{L}^4){\epsilon }_t,$$ specify 'MALags',1,'SMALags',4.
Seasonality	Seasonal periodicity, s	To specify the degree of seasonal integration s in the seasonal differencing polynomial Δs = 1 – Ls. For example, to specify the periodicity for seasonal integration of quarterly data, specify 'Seasonality',4. By default, Seasonality has value 0 (meaning no periodicity nor seasonal integration).

Note

You cannot assign values to the properties P and Q. For multiplicative ARIMA error models,

• regARIMA sets P equal to p + D + p_s + s.

• regARIMA sets Q equal to q + q_s

Specify Linear Regression Models Using Econometric Modeler App

You can specify the predictor variables in the regression component, and the error model lag structure and innovation distribution, using the Econometric Modeler app. The app treats all coefficients as unknown and estimable.

At the command line, open the Econometric Modeler app.

econometricModeler

Alternatively, open the app from the apps gallery (see Econometric Modeler).

In the app, you can see all supported models by selecting a time series variable for the response in the Time Series pane. Then, on the Econometric Modeler tab, in the Models section, click the arrow to display the models gallery.

The Regression Models section contains supported regression models. To specify a multiple linear regression (MLR) model, select MLR. To specify regression models with ARMA errors, select RegARMA.

After you select a model, the app displays the Type Model Parameters dialog box, where Type is the model type. This figure shows the RegARMA Model Parameters dialog box.

Adjustable parameters depend on the model Type. In general, adjustable parameters include:

• Predictor variables for the linear regression component, listed in the Predictors section.

  • For regression models with ARMA errors, you must include at least one predictor in the model. To include a predictor, select the corresponding check box in the Include? column.

  • For MLR models, you can clear all check boxes in the Include? column. In this case, you can specify a constant mean model (intercept-only model) by selecting the Include Intercept check box. Or, you can specify an error-only model by clearing the Include Intercept check box.

• The innovation distribution and nonseasonal lags for the error model, for regression models with ARMA errors.

As you adjust parameter values, the equation in the Model Equation section changes to match your specifications. Adjustable parameters correspond to input and name-value pair arguments described in the previous sections and in the regARIMA reference page.

For more details on specifying models using the app, see Fit Models to Data and Specifying Univariate Lag Operator Polynomials Interactively.

What Are Regression Models with Time Series Errors

Regression models with time series errors attempt to explain the mean behavior of a response series (y_t, t = 1,...,T) by accounting for linear effects of predictors (X_t) using a multiple linear regression (MLR). However, the errors (u_t), called unconditional disturbances, are time series rather than white noise, which is a departure from the linear model assumptions. Unlike the ARIMA model that includes exogenous predictors, regression models with time series errors preserve the sensitivity interpretation of the regression coefficients (β) [3].

These models are particularly useful for econometric data. Use these models to:

• Analyze the effects of a new policy on a market indicator (an intervention model).

• Forecast population size adjusting for predictor effects, such as expected prevalence of a disease.

• Study the behavior of a process adjusting for calendar effects. For example, you can analyze traffic volume by adjusting for the effects of major holidays. For details, see [4].

• Estimate the trend by including time (t) in the model.

• Forecast total energy consumption accounting for current and past prices of oil and electricity (distributed lag model).

Use these tools in Econometrics Toolbox™ to:

• Specify a regression model with ARIMA errors (see regARIMA).

• Estimate parameters using a specified model, and response and predictor data (see estimate).

• Simulate responses using a model and predictor data (see simulate).

• Forecast responses using a model and future predictor data (see forecast).

• Infer residuals and estimated unconditional disturbances from a model using the model and predictor data (see infer).

• filter innovations through a model using the model and predictor data

• Generate impulse responses (see impulse).

• Compare a regression model with ARIMA errors to an ARIMAX model (see arima).

A regression model with time series errors has the following form (in lag operator notation):

$$\begin{array}{c}{y}_t=c+X_t\beta +u_t\\ a\left(L\right)A\left(L\right){\left(1-L\right)}^D\left(1-L^s\right)u_t=b\left(L\right)B\left(L\right){\epsilon }_t,\end{array}$$

(4)

where:

• t = 1,...,T.

• y_t is the response series.

• X_t is row t of X, which is the matrix of concatenated predictor data vectors. That is, X_t is observation t of each predictor series.

• c is the regression model intercept.

• β is the regression coefficient.

• u_t is the disturbance series.

• ε_t is the innovations series.

• $$L^j{y}_t=y_{t-j}.$$

• $$a\left(L\right)=\left(1-a_{1}L-\mathrm{...}-a_p{L}^p\right),$$ which is the degree p, nonseasonal autoregressive polynomial.

• $$A\left(L\right)=\left(1-A_{1}L-\mathrm{...}-A_{p_s}{L}^{p_s}\right),$$ which is the degree p_s, seasonal autoregressive polynomial.

• $${\left(1-L\right)}^D,$$ which is the degree D, nonseasonal integration polynomial.

• $$\left(1-L^s\right),$$ which is the degree s, seasonal integration polynomial.

• $$b\left(L\right)=\left(1+b_{1}L+\mathrm{...}+b_q{L}^q\right),$$ which is the degree q, nonseasonal moving average polynomial.

• $$B\left(L\right)=\left(1+B_{1}L+\mathrm{...}+B_{q_s}{L}^{q_s}\right),$$ which is the degree q_s, seasonal moving average polynomial.

Following Box and Jenkins methodology, u_t is a stationary or unit root nonstationary, regular, linear time series. However, if u_t is unit root nonstationary, then you do not have to explicitly difference the series as they recommend in [2]. You can simply specify the seasonal and nonseasonal integration degree using the software.

Another deviation from the Box and Jenkins methodology is that u_t does not have a constant term (conditional mean), and therefore its unconditional mean is 0. However, the regression model contains an intercept term, c.

Note

If the unconditional disturbance process is nonstationary (i.e., the nonseasonal or seasonal integration degree is greater than 0), then the regression intercept, c, is not identifiable. For details, see Intercept Identifiability in Regression Models with ARIMA Errors.

The software enforces stability and invertibility of the ARMA process. That is,

$$\psi (L)=\frac{b(L)B(L)}{a(L)A(L)}=1+{\psi }_{1}L+{\psi }_2{L}^2+\mathrm{...},$$

where the series {ψ_t} must be absolutely summable. The conditions for {ψ_t} to be absolutely summable are:

• a(L) and A(L) are stable (i.e., the eigenvalues of a(L) = 0 and A(L) = 0 lie inside the unit circle).

• b(L) and B(L) are invertible (i.e., their eigenvalues lie of b(L) = 0 and B(L) = 0 inside the unit circle).

The software uses maximum likelihood for parameter estimation. You can choose either a Gaussian or Student’s t distribution for the innovations, ε_t.

The software treats predictors as nonstochastic variables for estimation and inference.

What Are Time Series Regression Models?

Time series regression models attempt to explain the current response using the response history (autoregressive dynamics) and the transfer of dynamics from relevant predictors (or otherwise). Theoretical frameworks for potential relationships among variables often permit different representations of the system.

Use time series regression models to analyze time series data, which are measurements that you take at successive time points. For example, use time series regression modeling to:

• Examine the linear effects of the current and past unemployment rates and past inflation rates on the current inflation rate.

• Forecast GDP growth rates by using an ARIMA model and include the CPI growth rate as a predictor.

• Determine how a unit increase in rainfall, amount of fertilizer, and labor affect crop yield.

You can start a time series analysis by building a design matrix (X_t), which can include current and past observations of predictors. You can also complement the regression component with an autoregressive (AR) component to account for the possibility of response (y_t) dynamics. For example, include past measurements of inflation rate in the regression component to explain the current inflation rate. AR terms account for dynamics unexplained by the regression component, which is necessarily underspecified in econometric applications. Also, the AR terms absorb residual autocorrelations, simplify innovation models, and generally improve forecast performance. Then, apply ordinary least squares (OLS) to the multiple linear regression (MLR) model:

$$y_t=X_t\beta +u_t.$$

If a residual analysis suggests classical linear model assumption departures such as that heteroscedasticity or autocorrelation (i.e., nonspherical errors), then:

• You can estimate robust HAC (heteroscedasticity and autocorrelation consistent) standard errors (for details, see hac).

• If you know the innovation covariance matrix (at least up to a scaling factor), then you can apply generalized least squares (GLS). Given that the innovation covariance matrix is correct, GLS effectively reduces the problem to a linear regression where the residuals have covariance I.

• If you do not know the structure of the innovation covariance matrix, but know the nature of the heteroscedasticity and autocorrelation, then you can apply feasible generalized least squares (FGLS). FGLS applies GLS iteratively, but uses the estimated residual covariance matrix. FGLS estimators are efficient under certain conditions. For details, see [1], Chapter 11.

There are time series models that model the dynamics more explicitly than MLR models. These models can account for AR and predictor effects as with MLR models, but have the added benefits of:

• Accounting for moving average (MA) effects. Include MA terms to reduce the number of AR lags, effectively reducing the number of observation required to initialize the model.

• Easily modeling seasonal effects. In order to model seasonal effects with an MLR model, you have to build an indicator design matrix.

• Modeling nonseasonal and seasonal integration for unit root nonstationary processes.

These models also differ from MLR in that they rely on distribution assumptions (i.e., they use maximum likelihood for estimation). Popular types of time series regression models include:

• Autoregressive integrated moving average with exogenous predictors (ARIMAX). This is an ARIMA model that linearly includes predictors (exogenous or otherwise). For details, see arima or ARIMAX(p,D,q) Model.

• Regression model with ARIMA time series errors. This is an MLR model where the unconditional disturbance process (u_t) is an ARIMA time series. In other words, you explicitly model u_t as a linear time series. For details, see regARIMA.

• Distributed lag model (DLM). This is an MLR model that includes the effects of predictors that persist over time. In other words, the regression component contains coefficients for contemporaneous and lagged values of predictors. Econometrics Toolbox does not contain functions that model DLMs explicitly, but you can use regARIMA or fitlm with an appropriately constructed predictor (design) matrix to analyze a DLM.

• Transfer function (autoregressive distributed lag) model. This model extends the distributed lag framework in that it includes autoregressive terms (lagged responses). Econometrics Toolbox does not contain functions that model DLMs explicitly, but you can use the arima functionality with an appropriately constructed predictor matrix to analyze an autoregressive DLM.

The choice you make on which model to use depends on your goals for the analysis, and the properties of the data.

References

[1] Greene, William. H. Econometric Analysis. 6th ed. Upper Saddle River, NJ: Prentice Hall, 2008.

[2] Box, George E. P., Gwilym M. Jenkins, and Gregory C. Reinsel. Time Series Analysis: Forecasting and Control. 3rd ed. Englewood Cliffs, NJ: Prentice Hall, 1994.

[3] Hyndman, Rob. J. (2010, October). “The ARIMAX Model Muddle.” Rob J. Hyndman. Retrieved May 4, 2017 from https://robjhyndman.com/hyndsight/arimax/.

[4] Tsay, Ruey S. "Regression Models with Time Series Errors." Journal of the American Statistical Association 79 (March 1984): 118–124. https://doi.org/10.1080/01621459.1984.10477073.