# Setup

This question is a followup on this question. I'm interested in the asymptotic distribution of certain quadratic forms.

So, let $$Z$$ be a $$p$$-dimensional random vector with (unknown) distribution $$P$$. Consider a mapping $$g:\mathbb R^q \times \mathbb R^p \rightarrow \mathbb R^k$$. Let $$\theta^*\in \mathbb R^p$$ satisfy $$\mathbb E_P[g(Z;\theta^*)] = 0_k$$.

# Question 1

Under very general regularity conditions on $$g$$, what can be said about the asymptotic distribution of the (random) scalar quantity $$\alpha_n(\theta) := n\mathbb E_{\hat{P}_n}[g(Z;\theta)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\theta)]?$$ Here, $$\mathbb E_{\hat{P}_n}[g(Z;\theta)] = (1/n)\sum_{i=1}^ng(z_i;\theta) \in \mathbb R^k$$ is the empirical mean of the random vector $$g(Z;\theta)$$ from an i.i.d sample $$z_1,\ldots,z_n\sim P$$, and $$\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)] \in \mathbb R^{k \times k}$$ is the empirical covariance matrix.

# Question 2: specialization to MLE

Same question with $$g(Z;\theta) := -\partial_\theta \log p_\theta(Z)$$, the score function for a parametric family of distributions (densities thereof) $$\{p_\theta\}$$, assumed to contain the true distribution of $$Z$$, namely $$P$$.

# Question 3: specialization to location parameter estimation

Same question with $$g(Z;\theta) = Z - \theta$$.

# Observations

My wild guesses are

• $$\alpha_n$$ converges in distribution to some chi-squared distribution.
• The solution of the problem will come from an application of the (functional) delta method.
• Turns out the $\alpha_n(\theta)$ matches the so-called empirical likelihood functional (Owen 1990, Qin & Lawless 1994). If $\hat{\theta}_n$ minimizes $\alpha_n(\theta)$, then $\alpha_n(\hat{\theta}_n)-\alpha_n(\theta^*) \rightsquigarrow \chi^2_{(p)}$. This is theorem 2 of Qin & Lawless 1994. It's not much of an insight, but I don't know if I should add this as an "observation" under the question or a separate answer. – dohmatob Nov 5 '18 at 12:16
• If $\theta$ is fixed, then this question was answered, at the end of the answer linked in your post -- just replace $z_i$ there by $g(z_i;\theta)$. – Iosif Pinelis Nov 13 '18 at 19:00
• @IosifPinelis Indeed. I thought of it but it was too good to be true. Thanks again. – dohmatob Nov 13 '18 at 20:09
• BTW, using your bounds I get, $$R_n(\theta) \rightsquigarrow \begin{cases}\frac{1}{n}\chi^2_{(r)},&\mbox{ if }\mathbb E_{P}[g(Z;\theta)] = 0,\\\mathbb E_{P}[\|g(Z;\theta)\|^2] + \mathcal N(0,\frac{C}{n}),&\mbox{ else,}\end{cases}$$ where $r=r(\theta)$ is the rank of the covariance matrix of $g(Z;\theta)$ and $C \ge 0$ is an absolute constant. – dohmatob Nov 13 '18 at 20:44