Setup

This question is a followup on this question. I'm interested in the asymptotic distribution of certain quadratic forms.

So, let $Z$ be a $p$-dimensional random vector with (unknown) distribution $P$. Consider a mapping $g:\mathbb R^q \times \mathbb R^p \rightarrow \mathbb R^k$. Let $\theta^*\in \mathbb R^p$ satisfy $\mathbb E_P[g(Z;\theta^*)] = 0_k$.

Question 1

Under very general regularity conditions on $g$, what can be said about the asymptotic distribution of the (random) scalar quantity $$\alpha_n(\theta) := n\mathbb E_{\hat{P}_n}[g(Z;\theta)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\theta)]? $$ Here, $\mathbb E_{\hat{P}_n}[g(Z;\theta)] = (1/n)\sum_{i=1}^ng(z_i;\theta) \in \mathbb R^k$ is the empirical mean of the random vector $g(Z;\theta)$ from an i.i.d sample $z_1,\ldots,z_n\sim P$, and $\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)] \in \mathbb R^{k \times k}$ is the empirical covariance matrix.

Question 2: specialization to MLE

Same question with $g(Z;\theta) := -\partial_\theta \log p_\theta(Z)$, the score function for a parametric family of distributions (densities thereof) $\{p_\theta\}$, assumed to contain the true distribution of $Z$, namely $P$.

Question 3: specialization to location parameter estimation

Same question with $g(Z;\theta) = Z - \theta$.

Observations

My wild guesses are

• $\alpha_n$ converges in distribution to some chi-squared distribution.
• The solution of the problem will come from an application of the (functional) delta method.