diff --git a/main.tex b/main.tex index 32f3e9c..a268864 100644 --- a/main.tex +++ b/main.tex @@ -180,7 +180,7 @@ mathematically formulated by strict stationarity. \end{tabular} \subsubsection{Weak} \label{weak-stationarity} - It is impossible to "prove" the theoretical concept of stationarity from data. We can only search for evidence in favor or against it. \\ + It is impossible to «prove» the theoretical concept of stationarity from data. We can only search for evidence in favor or against it. \\ \vspace{0.1cm} However, with strict stationarity, even finding evidence only is too difficult. We thus resort to the concept of weak stationarity. @@ -677,7 +677,7 @@ Call: arima(x = log(lynx), order = c(2, 0, 0)) \end{itemize} \subsection{Model diagnostics} -\subsubsection{Residual analysis} +\subsubsection{Residual analysis}\label{residual-analysis} "residuals" = "estimated innovations" $$\hat{E_t} = (y_t - \hat{m}) - (\hat{\alpha_1}(y_{t-1} - \hat{m}) - \dots - \hat{\alpha}_p(y_{t-1} - \hat{m}))$$ With assumptions as in Chapter \ref{ar-1} \\ @@ -727,6 +727,91 @@ in the sense that the last q innovation terms $E_{t-1} , E_{t-2} ,...$ are inclu $$X_t = E_t + \beta_1 E_{t-1} + \beta_2 E_{t-2} + \dots + \beta_q E_{t-q}$$ This is a time series process that is stationary, but not i.i.d. In many aspects, $MA(q)$ models are complementary to $AR(p)$. +\subsubsection{Stationarity of MA models} +We first restrict ourselves to the simple $MA(1)$-model: +$$X_t = E_t + \beta_1 E_{t-1}$$ +The series $X_t$ is always weakly stationary, no matter what the choice of the parameter $\beta_1$ is. + +\subsubsection{ACF/PACF of MA processes} +For the ACF +$$\rho(1) = \frac{\gamma(1)}{\gamma(0)} = \frac{\beta_1}{1+\beta_1^2} < 0.5$$ +and +$$\rho(k) = 0 \, \forall \, k > 1$$ + +Thus, we have a «cut-off» situation, i.e. a similar behavior to the one of the PACF in an $AR(1)$ process. This is why and how $AR(1)$ and $MA(1)$ are complementary. + +\subsubsection{Invertibility} +Without additional assumptions, the ACF of an $MA(1)$ does not allow identification of the generating model. +$$X_t = E_t + 0.5 E_{t-1}$$ +$$U_t = E_t + 2 E_{t-1}$$ +have identical ACF! +$$\rho(1) = \frac{\beta_{1}}{1+\beta_1^2} = \frac{1/\beta_1}{1+(1/\beta_1^2)}$$ + +\begin{itemize} + \item An $MA(1)$-, or in general an $MA(q)$-process is said to be invertible if the roots of the characteristic polynomial $\Theta(B)$ exceed one in absolute value. + \item Under this condition, there exists only one $MA(q)$-process for any given ACF. But please note that any $MA(q)$ is stationary, no matter if it is invertible or not. + \item The condition on the characteristic polynomial translates to restrictions on the coefficients. For any MA(1)-model, $|\beta_1| < 1$ is required. + \item R function \verb|polyroot()| can be used for finding the roots. +\end{itemize} + +\textbf{Practical importance:} \\ +The condition of invertibility is not only a technical issue, but has important practical meaning. All invertible $MA(q)$ processes can be expressed in terms of an $AR(\infty)$, e.g. for an $MA(1)$: +\begin{align*} +X_t &= E_t + \beta_1 E_{t-1} \\ + &= E_t + \beta_1(X_{t-1} - \beta_1 E_{t-2}) \\ + &= \dots \\ + &= E_t + \beta_1 X_{t-1} - \beta_1^2 X_{t-2} + \beta_1^3X_{t-3} + \dots \\ + &= E_t + \sum_{i=1}^\infty \psi_i X_{t-i} +\end{align*} + +\subsection{Fitting MA(q)-models to data} +As with AR(p) models, there are three main steps: +\begin{enumerate} + \item Model identification + \begin{itemize} + \item Is the series stationary? + \item Do the properties of ACF/PACF match? + \item Derive order $q$ from the cut-off in the ACF + \end{itemize} + \item Parameter estimation + \begin{itemize} + \item How to determine estimates for $m, \beta_1 ,\dots, \beta_q, \sigma_E^2$? + \item Conditional Sum of Squares or MLE + \end{itemize} + \item Model diagnostics + \begin{itemize} + \item With the same tools/techniques as for AR(p) models + \end{itemize} +\end{enumerate} + +\subsubsection{Parameter estimation} +The simplest idea is to exploit the relation between model parameters and autocorrelation coefficients («Yule-Walker») after the global mean $m$ has been estimated and subtracted. \\ +In contrast to the Yule-Walker method for AR(p) models, this yields an inefficient estimator that generally generates poor results and hence should not be used in practice. + +\vspace{.2cm} +It is better to use \textbf{Conditional sum of squares}:\\ +This is based on the fundamental idea of expressing $\sum E_t^2$ in terms of $X_1 ,..., X_n$ and $\beta_1 ,\dots, \beta_q$, as the innovations themselves are unobservable. This is possible for any invertible $MA(q)$, e.g. the $MA(1)$: +$$E_t = X_t = \beta_1 X_{t-1} + \beta_1^2 X_{t-2} + \dots + (-\beta)^{t-1} X_1 + \beta_1^t E_0$$ +Conditional on the assumption of $E_0 = 0$ , it is possible to rewrite $\sum E_t^2$ for any $MA(1)$ using $X_1 ,\dots, X_n $ and $\beta_1$. \\ +\vspace{.2cm} +Numerical optimization is required for finding the optimal parameter $\beta_1$, but is available in R function \verb|arima()| with: +\begin{lstlisting}[language=R] +> arima(..., order=c(...), method="CSS") +\end{lstlisting} + +\textbf{Maximium-likelihood estimation} +\begin{lstlisting}[language=R] +> arima(..., order=c(...), method="CSS-ML") +\end{lstlisting} +This is the default methods in R, which is based on finding starting values for MLE using the CSS approach. If assuming Gaussian innovations, then: +$$X_t = E_t + \beta_1 E_{t-1} + \beta_q E_{t-q}$$ +will follow a Gaussian distribution as well, and we have: +$$X = (X_1, \dots, X_n) \sim N(0,V)$$ +Hence it is possible to derive the likelihood function and simultaneously estimate the parameters $m;\beta_1,\dots,\beta_q;\sigma_E^2$. + +\subsubsection{Residual analysis} +See \ref{residual-analysis} + \scriptsize