1
$\begingroup$

Under very general conditions on the random $p$-dimensional vector $Z$, what can be said about the asymptotic distribution of the (random) scalar quantity $R_n := \mathbb E_{\hat{P}_n}[Z]^T\operatorname{Cov}_{\hat{P}_n}[Z]^{-1}\mathbb E_{\hat{P}_n}[Z]$ ?

Here, $\mathbb E_{\hat{P}_n}[Z] = (1/n)\sum_{i=1}^nz_i \in \mathbb R^p$ is the empirical mean of $Z$ from an i.i.d sample $z_1,\ldots,z_n$, and $\operatorname{Cov}_{\hat{P}_n}[Z] \in \mathbb R^{p \times p}$ is the empirical covariance matrix.

Notes

My ultimate goal is to understand the rate of growth of $R_n$ as a function of $n$.

Observations

My wild guess is that $R_n$ should be "concentrated" around $\mu^T\Sigma^{-1}\mu$ where $\mu \in \mathbb R^p$ is the mean of $Z$ and $\Sigma \in \mathbb R^{p \times p}$ is its covariance matrix.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

If $\mu\ne0$, then the distribution of $R_n$ is asymptotically normal with asymptotic mean $\mu^T\Sigma^{-1}\mu$ and an explicit asymptotic variance $\tilde\sigma^2/n$; see e.g. Theorem 3.9, page 1018, where a bound on the rate of convergence is also given. More specifically, \begin{equation} \tilde\sigma^2=EL(V)^2=E(2\xi-\xi^2+\mu^T\Sigma^{-1}\mu)^2, \end{equation} where \begin{equation} V:=(Y-EY,(Y-EY)(Y-EY)^T-I),\quad Y:=\Sigma^{-1/2}z_1, \end{equation} \begin{equation} L(x_1,x_2):=2x_1^T\,EY-EY^T\,x_2\,EY, \end{equation} \begin{equation} \xi:=EY^T\,(Y-EY)=\mu^T\Sigma^{-1}(z_1-\mu). \end{equation}

If $\mu=0$, then the distribution of properly normalized $R_n$ is asymptotically chi-squared; see e.g. Theorem 3, page 48.

Improve this answer
$\endgroup$
10
  • $\begingroup$ Two short questions: (1) In Theorem 3.9 what's an explict formula for $\tilde{\sigma}$ (it's supposed to be defined in 2.21, but I kinda lost tract of the definition of $L(V)$ relative to Hotelling's statistic under study in Theorem 3.9. (2) Can the works in this paper be applied to get asymptotic distributions of $M$-estimators ? Thanks in advance. $\endgroup$
    – dohmatob
    Commented Oct 23, 2018 at 23:06
  • $\begingroup$ (1) I have now reproduced the explicit expression for $\tilde\sigma$. (2) Results of the first referenced paper do apply to $M$-estimators, albeit somewhat indirectly; see the comment right after Remark 7.3 in projecteuclid.org/euclid.ejs/1491897618 . $\endgroup$
    – Iosif Pinelis
    Commented Oct 24, 2018 at 15:21
  • $\begingroup$ Thanks for the generous update. OK, I see . However, concerning $M$-estimators, this extension is not completely trivial though. IMO, it deserves a separate treatment (post, manuscript etc.). I don't master enough statistics to do this myself without screwing certain things badly. I can make this into a separate question, and maybe you could roughly sketch the main steps ? Thanks. $\endgroup$
    – dohmatob
    Commented Oct 24, 2018 at 16:37
  • $\begingroup$ I thought the comment right after Remark 7.3 in the MLE paper should be enough. I guess I can try to state the result for $M$-estimators formally. However, I think this would rather be a separate question. $\endgroup$
    – Iosif Pinelis
    Commented Oct 24, 2018 at 19:55
  • 1
    $\begingroup$ If $\Sigma$ is not full rank, it just means that the underlying linear space was wrongly chosen: it should be the column space (say $V$) of $\Sigma$, preferably with a good basis of it, say an orthonormal eigenbasis or somewhat close to it. Then the matrix of the linear transformation $x\mapsto\Sigma x$ in that good basis will be somewhat close to a diagonal matrix, or even to the identity matrix after appropriate re-scaling. So, if $\mu\in V$, then we will have $\mu^T\Sigma^{-1}\mu\in[0,\infty)$. If $\mu\notin V$, then $\mu^T\Sigma^{-1}\mu$ should naturally be defined as $\infty$. $\endgroup$
    – Iosif Pinelis
    Commented Nov 14, 2018 at 14:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .