Under very general conditions on the random $p$-dimensional vector $Z$, what can be said about the asymptotic distribution of the (random) scalar quantity $R_n := \mathbb E_{\hat{P}_n}[Z]^T\operatorname{Cov}_{\hat{P}_n}[Z]^{-1}\mathbb E_{\hat{P}_n}[Z]$ ?

Here, $\mathbb E_{\hat{P}_n}[Z] = (1/n)\sum_{i=1}^nz_i \in \mathbb R^p$ is the empirical mean of $Z$ from an i.i.d sample $z_1,\ldots,z_n$, and $\operatorname{Cov}_{\hat{P}_n}[Z] \in \mathbb R^{p \times p}$ is the empirical covariance matrix.


My ultimate goal is to understand the rate of growth of $R_n$ as a function of $n$.


My wild guess is that $R_n$ should be "concentrated" around $\mu^T\Sigma^{-1}\mu$ where $\mu \in \mathbb R^p$ is the mean of $Z$ and $\Sigma \in \mathbb R^{p \times p}$ is its covariance matrix.


If $\mu\ne0$, then the distribution of $R_n$ is asymptotically normal with asymptotic mean $\mu^T\Sigma^{-1}\mu$ and an explicit asymptotic variance $\tilde\sigma^2/n$; see e.g. Theorem 3.9, page 1018, where a bound on the rate of convergence is also given. More specifically, \begin{equation} \tilde\sigma^2=EL(V)^2=E(2\xi-\xi^2+\mu^T\Sigma^{-1}\mu)^2, \end{equation} where \begin{equation} V:=(Y-EY,(Y-EY)(Y-EY)^T-I),\quad Y:=\Sigma^{-1/2}z_1, \end{equation} \begin{equation} L(x_1,x_2):=2x_1^T\,EY-EY^T\,x_2\,EY, \end{equation} \begin{equation} \xi:=EY^T\,(Y-EY)=\mu^T\Sigma^{-1}(z_1-\mu). \end{equation}

If $\mu=0$, then the distribution of properly normalized $R_n$ is asymptotically chi-squared; see e.g. Theorem 3, page 48.

  • $\begingroup$ Two short questions: (1) In Theorem 3.9 what's an explict formula for $\tilde{\sigma}$ (it's supposed to be defined in 2.21, but I kinda lost tract of the definition of $L(V)$ relative to Hotelling's statistic under study in Theorem 3.9. (2) Can the works in this paper be applied to get asymptotic distributions of $M$-estimators ? Thanks in advance. $\endgroup$ – dohmatob Oct 23 '18 at 23:06
  • $\begingroup$ (1) I have now reproduced the explicit expression for $\tilde\sigma$. (2) Results of the first referenced paper do apply to $M$-estimators, albeit somewhat indirectly; see the comment right after Remark 7.3 in projecteuclid.org/euclid.ejs/1491897618 . $\endgroup$ – Iosif Pinelis Oct 24 '18 at 15:21
  • $\begingroup$ Thanks for the generous update. OK, I see . However, concerning $M$-estimators, this extension is not completely trivial though. IMO, it deserves a separate treatment (post, manuscript etc.). I don't master enough statistics to do this myself without screwing certain things badly. I can make this into a separate question, and maybe you could roughly sketch the main steps ? Thanks. $\endgroup$ – dohmatob Oct 24 '18 at 16:37
  • $\begingroup$ I thought the comment right after Remark 7.3 in the MLE paper should be enough. I guess I can try to state the result for $M$-estimators formally. However, I think this would rather be a separate question. $\endgroup$ – Iosif Pinelis Oct 24 '18 at 19:55
  • 1
    $\begingroup$ If $\Sigma$ is not full rank, it just means that the underlying linear space was wrongly chosen: it should be the column space (say $V$) of $\Sigma$, preferably with a good basis of it, say an orthonormal eigenbasis or somewhat close to it. Then the matrix of the linear transformation $x\mapsto\Sigma x$ in that good basis will be somewhat close to a diagonal matrix, or even to the identity matrix after appropriate re-scaling. So, if $\mu\in V$, then we will have $\mu^T\Sigma^{-1}\mu\in[0,\infty)$. If $\mu\notin V$, then $\mu^T\Sigma^{-1}\mu$ should naturally be defined as $\infty$. $\endgroup$ – Iosif Pinelis Nov 14 '18 at 14:52

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.