0
$\begingroup$

Setup

This question is a followup on this question. I'm interested in the asymptotic distribution of certain quadratic forms.

So, let $Z$ be a $p$-dimensional random vector with (unknown) distribution $P$. Consider a mapping $g:\mathbb R^q \times \mathbb R^p \rightarrow \mathbb R^k$. Let $\theta^*\in \mathbb R^p$ satisfy $\mathbb E_P[g(Z;\theta^*)] = 0_k$.

Question 1

Under very general regularity conditions on $g$, what can be said about the asymptotic distribution of the (random) scalar quantity $$\alpha_n(\theta) := n\mathbb E_{\hat{P}_n}[g(Z;\theta)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\theta)]? $$ Here, $\mathbb E_{\hat{P}_n}[g(Z;\theta)] = (1/n)\sum_{i=1}^ng(z_i;\theta) \in \mathbb R^k$ is the empirical mean of the random vector $g(Z;\theta)$ from an i.i.d sample $z_1,\ldots,z_n\sim P$, and $\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)] \in \mathbb R^{k \times k}$ is the empirical covariance matrix.

Question 2: specialization to MLE

Same question with $g(Z;\theta) := -\partial_\theta \log p_\theta(Z)$, the score function for a parametric family of distributions (densities thereof) $\{p_\theta\}$, assumed to contain the true distribution of $Z$, namely $P$.

Question 3: specialization to location parameter estimation

Same question with $g(Z;\theta) = Z - \theta$.

Observations

My wild guesses are

  • $\alpha_n$ converges in distribution to some chi-squared distribution.
  • The solution of the problem will come from an application of the (functional) delta method.
Improve this question
$\endgroup$
4
  • $\begingroup$ Turns out the $\alpha_n(\theta)$ matches the so-called empirical likelihood functional (Owen 1990, Qin & Lawless 1994). If $\hat{\theta}_n$ minimizes $\alpha_n(\theta)$, then $\alpha_n(\hat{\theta}_n)-\alpha_n(\theta^*) \rightsquigarrow \chi^2_{(p)}$. This is theorem 2 of Qin & Lawless 1994. It's not much of an insight, but I don't know if I should add this as an "observation" under the question or a separate answer. $\endgroup$
    – dohmatob
    Commented Nov 5, 2018 at 12:16
  • 1
    $\begingroup$ If $\theta$ is fixed, then this question was answered, at the end of the answer linked in your post -- just replace $z_i$ there by $g(z_i;\theta)$. $\endgroup$
    – Iosif Pinelis
    Commented Nov 13, 2018 at 19:00
  • $\begingroup$ @IosifPinelis Indeed. I thought of it but it was too good to be true. Thanks again. $\endgroup$
    – dohmatob
    Commented Nov 13, 2018 at 20:09
  • $\begingroup$ BTW, using your bounds I get, $$R_n(\theta) \rightsquigarrow \begin{cases}\frac{1}{n}\chi^2_{(r)},&\mbox{ if }\mathbb E_{P}[g(Z;\theta)] = 0,\\\mathbb E_{P}[\|g(Z;\theta)\|^2] + \mathcal N(0,\frac{C}{n}),&\mbox{ else,}\end{cases}$$ where $r=r(\theta)$ is the rank of the covariance matrix of $g(Z;\theta)$ and $C \ge 0$ is an absolute constant. $\endgroup$
    – dohmatob
    Commented Nov 13, 2018 at 20:44

0

Reset to default

You must log in to answer this question.