MA-models

main.tex

@@ -180,7 +180,7 @@ mathematically formulated by strict stationarity.
 \end{tabular}
 
 \subsubsection{Weak} \label{weak-stationarity}
- It is impossible to "prove" the theoretical concept of stationarity from data. We can only search for evidence in favor or against it. \\
+ It is impossible to «prove» the theoretical concept of stationarity from data. We can only search for evidence in favor or against it. \\
 \vspace{0.1cm}
 However, with strict stationarity, even finding evidence only is too difficult. We thus resort to the concept of weak stationarity.
 
@@ -677,7 +677,7 @@ Call: arima(x = log(lynx), order = c(2, 0, 0))
 \end{itemize}
 
 \subsection{Model diagnostics}
-\subsubsection{Residual analysis}
+\subsubsection{Residual analysis}\label{residual-analysis}
 "residuals" = "estimated innovations"
 $$\hat{E_t} = (y_t - \hat{m}) - (\hat{\alpha_1}(y_{t-1} - \hat{m}) - \dots - \hat{\alpha}_p(y_{t-1} - \hat{m}))$$
 With assumptions as in Chapter \ref{ar-1} \\
@@ -727,6 +727,91 @@ in the sense that the last q innovation terms $E_{t-1} , E_{t-2} ,...$ are inclu
 $$X_t = E_t + \beta_1 E_{t-1} + \beta_2 E_{t-2} + \dots + \beta_q E_{t-q}$$
 This is a time series process that is stationary, but not i.i.d. In many aspects, $MA(q)$ models are complementary to $AR(p)$.
 
+\subsubsection{Stationarity of MA models}
+We first restrict ourselves to the simple $MA(1)$-model:
+$$X_t = E_t + \beta_1 E_{t-1}$$
+The series $X_t$ is always weakly stationary, no matter what the choice of the parameter $\beta_1$ is.
+
+\subsubsection{ACF/PACF of MA processes}
+For the ACF
+$$\rho(1) = \frac{\gamma(1)}{\gamma(0)} = \frac{\beta_1}{1+\beta_1^2} < 0.5$$
+and 
+$$\rho(k) = 0 \, \forall \, k > 1$$
+
+Thus, we have a «cut-off» situation, i.e. a similar behavior to the one of the PACF in an $AR(1)$ process. This is why and how $AR(1)$ and $MA(1)$ are complementary.
+
+\subsubsection{Invertibility}
+Without additional assumptions, the ACF of an $MA(1)$ does not allow identification of the generating model.
+$$X_t = E_t + 0.5 E_{t-1}$$
+$$U_t = E_t + 2 E_{t-1}$$
+have identical ACF!
+$$\rho(1) = \frac{\beta_{1}}{1+\beta_1^2} = \frac{1/\beta_1}{1+(1/\beta_1^2)}$$
+
+\begin{itemize}
+    \item An $MA(1)$-, or in general an $MA(q)$-process is said to be invertible if the roots of the characteristic polynomial $\Theta(B)$ exceed one in absolute value.
+    \item Under this condition, there exists only one $MA(q)$-process for any given ACF. But please note that any $MA(q)$ is stationary, no matter if it is invertible or not.
+    \item The condition on the characteristic polynomial translates to restrictions on the coefficients. For any MA(1)-model, $|\beta_1| < 1$ is required.
+    \item  R function \verb|polyroot()| can be used for finding the roots.  
+\end{itemize}
+
+\textbf{Practical importance:} \\
+The condition of invertibility is not only a technical issue, but has important practical meaning. All invertible $MA(q)$ processes can be expressed in terms of an $AR(\infty)$, e.g. for an $MA(1)$:
+\begin{align*}
+X_t &= E_t + \beta_1 E_{t-1} \\
+    &= E_t + \beta_1(X_{t-1} - \beta_1 E_{t-2}) \\
+    &= \dots \\
+    &= E_t + \beta_1 X_{t-1} - \beta_1^2 X_{t-2} + \beta_1^3X_{t-3} + \dots \\
+    &= E_t + \sum_{i=1}^\infty \psi_i X_{t-i}
+\end{align*}
+
+\subsection{Fitting MA(q)-models to data}
+As with AR(p) models, there are three main steps:
+\begin{enumerate}
+    \item Model identification
+    \begin{itemize}
+        \item Is the series stationary?
+        \item Do the properties of ACF/PACF match?
+        \item Derive order $q$ from the cut-off in the ACF
+    \end{itemize}
+    \item Parameter estimation
+    \begin{itemize}
+        \item How to determine estimates for $m, \beta_1 ,\dots, \beta_q, \sigma_E^2$?
+        \item Conditional Sum of Squares or MLE
+    \end{itemize}
+    \item Model diagnostics
+    \begin{itemize}
+        \item With the same tools/techniques as for AR(p) models
+    \end{itemize}
+\end{enumerate}
+
+\subsubsection{Parameter estimation}
+The simplest idea is to exploit the relation between model parameters and autocorrelation coefficients («Yule-Walker») after the global mean $m$ has been estimated and subtracted. \\
+In contrast to the Yule-Walker method for AR(p) models, this yields an inefficient estimator that generally generates poor results and hence should not be used in practice.
+
+\vspace{.2cm}
+It is better to use \textbf{Conditional sum of squares}:\\
+This is based on the fundamental idea of expressing $\sum E_t^2$ in terms of $X_1 ,..., X_n$ and $\beta_1 ,\dots, \beta_q$, as the innovations themselves are unobservable. This is possible for any invertible $MA(q)$, e.g. the $MA(1)$:
+$$E_t = X_t = \beta_1 X_{t-1} + \beta_1^2 X_{t-2} + \dots + (-\beta)^{t-1} X_1 + \beta_1^t E_0$$
+Conditional on the assumption of $E_0 = 0$ , it is possible to rewrite $\sum E_t^2$ for any $MA(1)$ using $X_1 ,\dots, X_n $ and $\beta_1$. \\
+\vspace{.2cm}
+Numerical optimization is required for finding the optimal parameter $\beta_1$, but is available in R function \verb|arima()| with:
+\begin{lstlisting}[language=R]
+> arima(..., order=c(...), method="CSS")
+\end{lstlisting}
+
+\textbf{Maximium-likelihood estimation}
+\begin{lstlisting}[language=R]
+> arima(..., order=c(...), method="CSS-ML")
+\end{lstlisting}
+This is the default methods in R, which is based on finding starting values for MLE using the CSS approach. If assuming Gaussian innovations, then:
+$$X_t = E_t + \beta_1 E_{t-1} + \beta_q E_{t-q}$$
+will follow a Gaussian distribution as well, and we have:
+$$X = (X_1, \dots, X_n) \sim N(0,V)$$
+Hence it is possible to derive the likelihood function and simultaneously estimate the parameters $m;\beta_1,\dots,\beta_q;\sigma_E^2$.
+
+\subsubsection{Residual analysis}
+See \ref{residual-analysis}
+
 
 \scriptsize