0
$\begingroup$

Setup

This question is a followup on this question. I'm interested in the asymptotic distribution of certain quadratic forms.

So, let $Z$ be a $p$-dimensional random vector with (unknown) distribution $P$. Consider a mapping $g:\mathbb R^q \times \mathbb R^p \rightarrow \mathbb R^k$. Let $\theta^*\in \mathbb R^p$ satisfy $\mathbb E_P[g(Z;\theta^*)] = 0_k$.

Question 1

Under very general regularity conditions on $g$, what can be said about the asymptotic distribution of the (random) scalar quantity $$\alpha_n(\theta) := n\mathbb E_{\hat{P}_n}[g(Z;\theta)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\theta)]? $$ Here, $\mathbb E_{\hat{P}_n}[g(Z;\theta)] = (1/n)\sum_{i=1}^ng(z_i;\theta) \in \mathbb R^k$ is the empirical mean of the random vector $g(Z;\theta)$ from an i.i.d sample $z_1,\ldots,z_n\sim P$, and $\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)] \in \mathbb R^{k \times k}$ is the empirical covariance matrix.

Question 2: specialization to MLE

Same question with $g(Z;\theta) := -\partial_\theta \log p_\theta(Z)$, the score function for a parametric family of distributions (densities thereof) $\{p_\theta\}$, assumed to contain the true distribution of $Z$, namely $P$.

Question 3: specialization to location parameter estimation

Same question with $g(Z;\theta) = Z - \theta$.

Observations

My wild guesses are

  • $\alpha_n$ converges in distribution to some chi-squared distribution.
  • The solution of the problem will come from an application of the (functional) delta method.
$\endgroup$
  • $\begingroup$ Turns out the $\alpha_n(\theta)$ matches the so-called empirical likelihood functional (Owen 1990, Qin & Lawless 1994). If $\hat{\theta}_n$ minimizes $\alpha_n(\theta)$, then $\alpha_n(\hat{\theta}_n)-\alpha_n(\theta^*) \rightsquigarrow \chi^2_{(p)}$. This is theorem 2 of Qin & Lawless 1994. It's not much of an insight, but I don't know if I should add this as an "observation" under the question or a separate answer. $\endgroup$ – dohmatob Nov 5 '18 at 12:16
  • 1
    $\begingroup$ If $\theta$ is fixed, then this question was answered, at the end of the answer linked in your post -- just replace $z_i$ there by $g(z_i;\theta)$. $\endgroup$ – Iosif Pinelis Nov 13 '18 at 19:00
  • $\begingroup$ @IosifPinelis Indeed. I thought of it but it was too good to be true. Thanks again. $\endgroup$ – dohmatob Nov 13 '18 at 20:09
  • $\begingroup$ BTW, using your bounds I get, $$R_n(\theta) \rightsquigarrow \begin{cases}\frac{1}{n}\chi^2_{(r)},&\mbox{ if }\mathbb E_{P}[g(Z;\theta)] = 0,\\\mathbb E_{P}[\|g(Z;\theta)\|^2] + \mathcal N(0,\frac{C}{n}),&\mbox{ else,}\end{cases}$$ where $r=r(\theta)$ is the rank of the covariance matrix of $g(Z;\theta)$ and $C \ge 0$ is an absolute constant. $\endgroup$ – dohmatob Nov 13 '18 at 20:44

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.