2
$\begingroup$

Consider the following autoregressive process with normal errors: \begin{equation}\label{7YlUV4i8nuO}\tag{I} y_t = \phi y_{t-1}+ u_t, \quad u_t \overset{iid}{\sim} N(0,\sigma^2) \end{equation} We can show: \begin{equation} y_1 \sim N\left(0,\frac{\sigma^2}{1-\phi^2}\right), \quad y_t|y_{t-1} \sim N\left(\phi y_{t-1}, \sigma^2 \right) \end{equation} Denote the density of $y_{1}$ as $f_{y_1}(y_1;\phi,\sigma^2)$ and the conditional density of $y_t|y_{t-1}$ as $f_{y_t|y_{t-1}}(y_t|y_{t-1};\phi,\sigma^2)$. Given $y_T,...,y_1$ a sample, the likelihood function, and the log-likelihood function are: $$ L(\phi,\sigma^2) = f_{y_1}(y_1;\phi,\sigma^2) \prod_{t=2}^T f_{y_t|y_{t-1}}(y_t|y_{t-1};\phi,\sigma^2)$$ $$\mathcal L(\phi,\sigma^2) = \log f_{y_1}(y_1;\phi,\sigma^2) + \sum_{t=2}^T \log f_{y_t|y_{t-1}}(y_t|y_{t-1};\phi,\sigma^2)$$ See here to obtain the density $f_{y_1}(y_1;\phi,\sigma^2)$. For more details of all derivations, the reader can refer to this note.

Now, I want to find the likelihood functions for a class of stochastic process with compound Poisson distribution, i.e., a generalization of this. Given $N \sim \hbox{Poisson}(\lambda)$ with $\lambda\geq 0$ and $(y_t)_{t\in \mathbb Z}$ the AR(1) process given in (\ref{7YlUV4i8nuO}), define $x_t = \sum_{j=0}^{N} y_{t;j}$ (note that $N$ does not depend on $t$). It is straightforward to show that: $$x_t = \phi x_{t-1} + v_t, \quad v_t:= \sum_{j=0}^{N} u_{t;j}$$ where $(v_t)_t$ turns out to be white noise.

First, suppose the deterministic case $N=n> 0$ (If $N=0$, by convention $\sum_{j=0}^0 y_{tj} =0$ ) . Denoting $ \sigma_n^2 := n\sigma^2$, we have: $$x_t = \phi x_{t-1} + v_t^n, \quad v_t^n:= \sum_{j=1}^{n} u_{t;j}\sim N(0, \sigma_n^2)$$ Setting $\theta= (\phi,\sigma^2)$, we have that $x_1 \sim N(0,\sigma_n^2/(1-\phi^2))$, i.e, its denisity is: \begin{equation}\label{2D85niRxr6}\tag{II} f^n_{x_1}(x_1;\theta)= \frac{1}{\sqrt{ 2 \pi \sigma_n^2/(1-\phi^2) }} e^{ -\tfrac{1}{2} x_1^2/\sigma_n^2/(1-\phi^2) } \end{equation} Moreover, we have that $x_t|x_{t-1} \sim N( \phi x_{t-1}, \sigma_n^2 )$, i.e., its density is: \begin{equation}\label{2D85niRxr62}\tag{III} f^n_{x_t|x_{t-1}}(x_t|x_{t-1};\theta)= \frac{1}{\sqrt{ 2 \pi \sigma_n^2 }} e^{ -\tfrac{1}{2} (x_t - \phi x_{t-1} )^2/ \sigma_n^2 } \end{equation} In order to obtain the likelihood function for the random $N \sim \hbox{Poisson}(\lambda)$ case, first note that $x_1 = \sum_{j=0}^N y_{1;j}$ and since $P[N=0]>0$, we have $P[x_1=0]> 0$. That is, we have a mixed distribution (a density of a mixed random variable is neither discrete nor continuous) and this can complicate things a bit (Read this post on this point). So, as a first exercise, let's assume that $N>0$. Setting $\mu = (\phi, \sigma^2, \lambda)= (\theta, \lambda)$, we have that: \begin{equation} \begin{aligned} f_{x_1}(x_1|N>0; \mu)&= \sum_{n=1}^\infty f_{x_1|N=n}(x_1|N=n;\mu)P(N=n)\\ & = \sum_{n=1}^\infty f^n_{x_1}(x_1;\theta) P(N=n)\\ \end{aligned} \end{equation} Analogously: \begin{equation} \begin{aligned} f_{x_t|x_{t-1}}(x_t|x_{t-1}, N>0; \mu)&= \sum_{n=1}^\infty f_{x_t|x_{t-1},N=n}(x_{t}|x_{t-1},N=n;\mu)P(N=n)\\ & = \sum_{n=1}^\infty f^n_{x_t|x_{t-1}}(x_t|x_{t-1};\theta) P(N=n) \end{aligned} \end{equation} Thus, given $x_T,...,x_1$, the likelihood function is: \begin{equation}\label{HK518fhA8lu}\tag{II} \begin{aligned} L(\mu| N>0)&=f_{x_1}(x_1| N>0; \mu) \prod_{t=2}^T f_{x_t|x_{t-1}}(x_t|x_{t-1}, N>0; \mu)\\ &= \sum_{n=1}^\infty f^n_{x_1}(x_1;\theta) P(N=n) \prod_{t=2}^T \sum_{n=1}^\infty f^n_{x_1}(x_1;\theta) P(N=n) \\ &= e^{-T\lambda} \left[ \sum_{n=1}^\infty f^n_{x_1}(x_1;\theta) \frac{\lambda^n}{n!} \right] \left[ \prod_{t=2}^T\sum_{n=1}^\infty f^n_{x_t|x_{t-1}}(x_t|x_{t-1};\theta) \frac{\lambda^n}{n!}\right]\\ \end{aligned} \end{equation} The log-likelihood is \begin{equation}\label{HK518fhA8lu2}\tag{IV} \mathcal L(\mu| N>0) = -T\lambda + \log \left( \sum_{n=1}^\infty f^n_{x_1}(x_1;\theta) \frac{\lambda^n}{n!} \right) + \sum_{t=2}^T \log\left(\sum_{n=1}^\infty f^n_{x_t|x_{t-1}}(x_t|x_{t-1};\theta) \frac{\lambda^n}{n!}\right) \end{equation} where $f^n_{x_1}(x_1;\theta)$ and $f^n_{x_t|x_{t-1}}(x_t|x_{t-1};\theta)$ are respectively given by (\ref{2D85niRxr6}) and (\ref{2D85niRxr62}).

The general case is much more dramatic. \begin{equation} \begin{aligned} f_{x_1}(x_1; \mu)&= \mathbf{1}_{[x_1 = 0]} (x_1) P(N = 0) + \mathbf{1}_{[x_1 \neq 0]}(x_1)\sum_{n=1}^\infty f_{x_1|N=n}(x_1|N=n;\mu)P(N=n)\\ & = \mathbf{1}_{[x_1 = 0]}(x_1) e^{-\lambda} + \mathbf{1}_{[x_1 \neq 0]}(x_1)\sum_{n=1}^\infty f^n_{x_1}(x_1;\theta) e^{-\lambda}\frac{\lambda^n}{n!}\\ &= e^{-\lambda} \left[\mathbf{1}_{[x_1 = 0]}(x_1) + \mathbf{1}_{[x_1 \neq 0]}(x_1)\sum_{n=1}^\infty f^n_{x_1}(x_1;\theta) \frac{\lambda^n}{n!}\right] %\frac{1}{\sqrt{ 2 \pi \Tilde \sigma_n^2 }} e^{ -\tfrac{1}{2} x_1^2/\Tilde \sigma_n^2 } \end{aligned} \end{equation} Analogously, defining $D= [x_t = \phi x_{t-1}]$: \begin{equation} \begin{aligned} f_{x_t|x_{t-1}}(x_t|x_{t-1}; \mu)&= \mathbf{1}_{D}(x_{t}) P(N = 0) + \mathbf{1}_{D^c}(x_{t})\sum_{n=1}^\infty f_{x_t|x_{t-1},N=n}(x_{t}|x_{t-1},N=n;\mu)P(N=n)\\ & = \mathbf{1}_{D}(x_{t}) e^{-\lambda} + \mathbf{1}_{D^c}(x_{t})\sum_{n=1}^\infty f^n_{x_t|x_{t-1}}(x_t|x_{t-1};\theta) e^{-\lambda}\frac{\lambda^n}{n!}\\ &= e^{-\lambda} \left[\mathbf{1}_{D}(x_{t}) + \mathbf{1}_{D^c}(x_{t})\sum_{n=1}^\infty f^n_{x_t|x_{t-1}}(x_t|x_{t-1};\theta) \frac{\lambda^n}{n!}\right] %\frac{1}{\sqrt{ 2 \pi \Tilde \sigma_n^2 }} e^{ -\tfrac{1}{2} x_1^2/\Tilde \sigma_n^2 } \end{aligned} \end{equation} Thus, given $x_T,...,x_1$, the likelihood function is: \begin{equation}\label{26Y7lHzJ3}\tag{V} L(\mu)= f_{x_1}(x_1; \mu) \prod_{t=2}^T f_{x_t|x_{t-1}}(x_t|x_{t-1}; \mu) \end{equation} I'm going to avoid making the respective substitutions so as not to load too many formulas and not write the log-likelihood function. So, I guess my questions boil down to:

  1. Is there any way to try to find a more user-friendly form of the likelihood in (\ref{26Y7lHzJ3}) or the log-likelihood function for the general case? More specifically, how to find the $\hat \mu= (\hat \lambda, \hat \phi, \hat \sigma^2)$ that maximizes such functions?

  2. If not, can we try to maximize the case $N>0$ given by $\mathcal L(\mu| N>0)$ above in (\ref{HK518fhA8lu2})? If we can't maximize directly, I thought about doing a truncation on $\mathcal L(\mu| N>0)$, i.e., finding $\hat \mu_M$ that maximizes \begin{equation} \begin{aligned} \mathcal L_M(\mu| N>0) = -T\lambda + \log \left( \sum_{n=1}^M f^n_{x_1}(x_1;\theta) \frac{\lambda^n}{n!} \right) + \sum_{t=2}^T \log\left(\sum_{n=1}^M f^n_{x_t|x_{t-1}}(x_t|x_{t-1};\theta) \frac{\lambda^n}{n!}\right)\\ \end{aligned} \end{equation} Perhaps this maximization is feasible and with luck, I might have: \begin{equation} \hat{\mu}_M \longrightarrow \hat \mu = \hbox{argmax}_{\mu} \mathcal L(\mu| N>0), \quad (M \to \infty) \end{equation} If this is not a good strategy, what to do?

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.