Skip to main content
MathOverflow

Return to Question

deleted 133 characters in body; edited title
Source Link
dohmatob
dohmatob
  • 6.9k
  • 1
  • 18
  • 76

Asymptotic distribution of $n\mathbb E_{\hat{P}_n}[g(Z;\theta_nZ;\theta)]^TCov_]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta_nZ;\theta)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\theta_nZ;\theta)]$

Setup

This question is a followup on this question. I'm interested in the asymptotic distribution of certain quadratic forms.

So, let $Z$ be a $p$-dimensional random vector with (unknown) distribution $P$. Consider a mapping $g:\mathbb R^q \times \mathbb R^p \rightarrow \mathbb R^k$. Let $\theta^*\in \mathbb R^p$ satisfy $\mathbb E_P[g(Z;\theta^*)] = 0_k$, and let $\hat{\theta}_n \in \mathbb R^p$ be a $\sqrt{n}$-consistent estimator of $\theta^*$.

Question 1

Under very general regularity conditions on $g$, what can be said about the asymptotic distribution of the (random) scalar quantity $$\alpha_n := n\mathbb E_{\hat{P}_n}[g(Z;\hat{\theta}_n)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\hat{\theta}_n)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\hat{\theta}_n)]? $$$$\alpha_n(\theta) := n\mathbb E_{\hat{P}_n}[g(Z;\theta)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\theta)]? $$ Here, $\mathbb E_{\hat{P}_n}[g(Z;\hat{\theta}_n)] = (1/n)\sum_{i=1}^ng(z_i;\hat{\theta}_n) \in \mathbb R^k$$\mathbb E_{\hat{P}_n}[g(Z;\theta)] = (1/n)\sum_{i=1}^ng(z_i;\theta) \in \mathbb R^k$ is the empirical mean of the random vector $g(Z;\theta)$ from an i.i.d sample $z_1,\ldots,z_n\sim P$, and $\operatorname{Cov}_{\hat{P}_n}[g(Z;\hat{\theta}_n)] \in \mathbb R^{k \times k}$$\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)] \in \mathbb R^{k \times k}$ is the empirical covariance matrix.

Question 2: specialization to MLE

Same question with $g(Z;\theta) := -\partial_\theta \log p_\theta(Z)$, the score function for a parametric family of distributions (densities thereof) $\{p_\theta\}$, assumed to contain the true distribution of $Z$, namely $P$.

Question 3: specialization to location parameter estimation

Same question with $g(Z;\theta) = Z - \theta$.

Observations

My wild guesses are

  • $\alpha_n$ converges in distribution to some chi-squared distribution.
  • The solution of the problem will come from an application of the (functional) delta method.

Asymptotic distribution of $n\mathbb E_{\hat{P}_n}[g(Z;\theta_n)]^TCov_{\hat{P}_n}[g(Z;\theta_n)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\theta_n)]$

Setup

This question is a followup on this question. I'm interested in the asymptotic distribution of certain quadratic forms.

So, let $Z$ be a $p$-dimensional random vector with (unknown) distribution $P$. Consider a mapping $g:\mathbb R^q \times \mathbb R^p \rightarrow \mathbb R^k$. Let $\theta^*\in \mathbb R^p$ satisfy $\mathbb E_P[g(Z;\theta^*)] = 0_k$, and let $\hat{\theta}_n \in \mathbb R^p$ be a $\sqrt{n}$-consistent estimator of $\theta^*$.

Question 1

Under very general regularity conditions on $g$, what can be said about the asymptotic distribution of the (random) scalar quantity $$\alpha_n := n\mathbb E_{\hat{P}_n}[g(Z;\hat{\theta}_n)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\hat{\theta}_n)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\hat{\theta}_n)]? $$ Here, $\mathbb E_{\hat{P}_n}[g(Z;\hat{\theta}_n)] = (1/n)\sum_{i=1}^ng(z_i;\hat{\theta}_n) \in \mathbb R^k$ is the empirical mean of the random vector $g(Z;\theta)$ from an i.i.d sample $z_1,\ldots,z_n\sim P$, and $\operatorname{Cov}_{\hat{P}_n}[g(Z;\hat{\theta}_n)] \in \mathbb R^{k \times k}$ is the empirical covariance matrix.

Question 2: specialization to MLE

Same question with $g(Z;\theta) := -\partial_\theta \log p_\theta(Z)$, the score function for a parametric family of distributions (densities thereof) $\{p_\theta\}$, assumed to contain the true distribution of $Z$, namely $P$.

Question 3: specialization to location parameter estimation

Same question with $g(Z;\theta) = Z - \theta$.

Observations

My wild guesses are

  • $\alpha_n$ converges in distribution to some chi-squared distribution.
  • The solution of the problem will come from an application of the (functional) delta method.

Asymptotic distribution of $n\mathbb E_{\hat{P}_n}[g(Z;\theta)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\theta)]$

Setup

This question is a followup on this question. I'm interested in the asymptotic distribution of certain quadratic forms.

So, let $Z$ be a $p$-dimensional random vector with (unknown) distribution $P$. Consider a mapping $g:\mathbb R^q \times \mathbb R^p \rightarrow \mathbb R^k$. Let $\theta^*\in \mathbb R^p$ satisfy $\mathbb E_P[g(Z;\theta^*)] = 0_k$.

Question 1

Under very general regularity conditions on $g$, what can be said about the asymptotic distribution of the (random) scalar quantity $$\alpha_n(\theta) := n\mathbb E_{\hat{P}_n}[g(Z;\theta)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\theta)]? $$ Here, $\mathbb E_{\hat{P}_n}[g(Z;\theta)] = (1/n)\sum_{i=1}^ng(z_i;\theta) \in \mathbb R^k$ is the empirical mean of the random vector $g(Z;\theta)$ from an i.i.d sample $z_1,\ldots,z_n\sim P$, and $\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)] \in \mathbb R^{k \times k}$ is the empirical covariance matrix.

Question 2: specialization to MLE

Same question with $g(Z;\theta) := -\partial_\theta \log p_\theta(Z)$, the score function for a parametric family of distributions (densities thereof) $\{p_\theta\}$, assumed to contain the true distribution of $Z$, namely $P$.

Question 3: specialization to location parameter estimation

Same question with $g(Z;\theta) = Z - \theta$.

Observations

My wild guesses are

  • $\alpha_n$ converges in distribution to some chi-squared distribution.
  • The solution of the problem will come from an application of the (functional) delta method.
added 48 characters in body
Source Link
dohmatob
dohmatob
  • 6.9k
  • 1
  • 18
  • 76

Setup

This question is a followup on this question. I'm interested in the asymptotic distribution of certain quadratic forms.

So, let $Z$ be a $p$-dimensional random vector with (unknown) distribution $P$. Consider a mapping $g:\mathbb R^q \times \mathbb R^p \rightarrow \mathbb R^k$. Let $\theta^*\in \mathbb R^p$ satisfy $\mathbb E_P[g(Z;\theta^*)] = 0_k$, and let $\theta_n \in \mathbb R^p$$\hat{\theta}_n \in \mathbb R^p$ be a $\sqrt{n}$-consistent estimator of $\theta^*$.

Question 1

Under very general regularity conditions on $g$, what can be said about the asymptotic distribution of the (random) scalar quantity $$\alpha_n := n\mathbb E_{\hat{P}_n}[g(Z;\theta_n)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta_n)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\theta_n)]? $$$$\alpha_n := n\mathbb E_{\hat{P}_n}[g(Z;\hat{\theta}_n)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\hat{\theta}_n)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\hat{\theta}_n)]? $$ Here, $\mathbb E_{\hat{P}_n}[g(Z;\theta)] = (1/n)\sum_{i=1}^ng(z_i;\theta) \in \mathbb R^k$$\mathbb E_{\hat{P}_n}[g(Z;\hat{\theta}_n)] = (1/n)\sum_{i=1}^ng(z_i;\hat{\theta}_n) \in \mathbb R^k$ is the empirical mean of the random vector $g(Z;\theta)$ from an i.i.d sample $z_1,\ldots,z_n\sim P$, and $\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)] \in \mathbb R^{k \times k}$$\operatorname{Cov}_{\hat{P}_n}[g(Z;\hat{\theta}_n)] \in \mathbb R^{k \times k}$ is the empirical covariance matrix.

Question 2: specialization to MLE

Same question with $g(Z;\theta) := -\partial_\theta \log p_\theta(Z)$, the score function for a parametric family of distributions (densities thereof) $\{p_\theta\}$, assumed to contain the true distribution of $Z$, namely $P$.

Question 3: specialization to location parameter estimation

Same question with $g(Z;\theta) = Z - \theta$.

Observations

My wild guesses are

  • $\alpha_n$ converges in distribution to some chi-squared distribution.
  • The solution of the problem will come from an application of the (functional) delta method.

Setup

This question is a followup on this question. I'm interested in the asymptotic distribution of certain quadratic forms.

So, let $Z$ be a $p$-dimensional random vector with (unknown) distribution $P$. Consider a mapping $g:\mathbb R^q \times \mathbb R^p \rightarrow \mathbb R^k$. Let $\theta^*\in \mathbb R^p$ satisfy $\mathbb E_P[g(Z;\theta^*)] = 0_k$, and let $\theta_n \in \mathbb R^p$ be a $\sqrt{n}$-consistent estimator of $\theta^*$.

Question 1

Under very general regularity conditions on $g$, what can be said about the asymptotic distribution of the (random) scalar quantity $$\alpha_n := n\mathbb E_{\hat{P}_n}[g(Z;\theta_n)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta_n)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\theta_n)]? $$ Here, $\mathbb E_{\hat{P}_n}[g(Z;\theta)] = (1/n)\sum_{i=1}^ng(z_i;\theta) \in \mathbb R^k$ is the empirical mean of the random vector $g(Z;\theta)$ from an i.i.d sample $z_1,\ldots,z_n\sim P$, and $\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)] \in \mathbb R^{k \times k}$ is the empirical covariance matrix.

Question 2: specialization to MLE

Same question with $g(Z;\theta) := -\partial_\theta \log p_\theta(Z)$, the score function for a parametric family of distributions (densities thereof) $\{p_\theta\}$, assumed to contain the true distribution of $Z$, namely $P$.

Question 3: specialization to location parameter estimation

Same question with $g(Z;\theta) = Z - \theta$.

Observations

My wild guesses are

  • $\alpha_n$ converges in distribution to some chi-squared distribution.
  • The solution of the problem will come from an application of the (functional) delta method.

Setup

This question is a followup on this question. I'm interested in the asymptotic distribution of certain quadratic forms.

So, let $Z$ be a $p$-dimensional random vector with (unknown) distribution $P$. Consider a mapping $g:\mathbb R^q \times \mathbb R^p \rightarrow \mathbb R^k$. Let $\theta^*\in \mathbb R^p$ satisfy $\mathbb E_P[g(Z;\theta^*)] = 0_k$, and let $\hat{\theta}_n \in \mathbb R^p$ be a $\sqrt{n}$-consistent estimator of $\theta^*$.

Question 1

Under very general regularity conditions on $g$, what can be said about the asymptotic distribution of the (random) scalar quantity $$\alpha_n := n\mathbb E_{\hat{P}_n}[g(Z;\hat{\theta}_n)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\hat{\theta}_n)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\hat{\theta}_n)]? $$ Here, $\mathbb E_{\hat{P}_n}[g(Z;\hat{\theta}_n)] = (1/n)\sum_{i=1}^ng(z_i;\hat{\theta}_n) \in \mathbb R^k$ is the empirical mean of the random vector $g(Z;\theta)$ from an i.i.d sample $z_1,\ldots,z_n\sim P$, and $\operatorname{Cov}_{\hat{P}_n}[g(Z;\hat{\theta}_n)] \in \mathbb R^{k \times k}$ is the empirical covariance matrix.

Question 2: specialization to MLE

Same question with $g(Z;\theta) := -\partial_\theta \log p_\theta(Z)$, the score function for a parametric family of distributions (densities thereof) $\{p_\theta\}$, assumed to contain the true distribution of $Z$, namely $P$.

Question 3: specialization to location parameter estimation

Same question with $g(Z;\theta) = Z - \theta$.

Observations

My wild guesses are

  • $\alpha_n$ converges in distribution to some chi-squared distribution.
  • The solution of the problem will come from an application of the (functional) delta method.
Source Link
dohmatob
dohmatob
  • 6.9k
  • 1
  • 18
  • 76

Asymptotic distribution of $n\mathbb E_{\hat{P}_n}[g(Z;\theta_n)]^TCov_{\hat{P}_n}[g(Z;\theta_n)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\theta_n)]$

Setup

This question is a followup on this question. I'm interested in the asymptotic distribution of certain quadratic forms.

So, let $Z$ be a $p$-dimensional random vector with (unknown) distribution $P$. Consider a mapping $g:\mathbb R^q \times \mathbb R^p \rightarrow \mathbb R^k$. Let $\theta^*\in \mathbb R^p$ satisfy $\mathbb E_P[g(Z;\theta^*)] = 0_k$, and let $\theta_n \in \mathbb R^p$ be a $\sqrt{n}$-consistent estimator of $\theta^*$.

Question 1

Under very general regularity conditions on $g$, what can be said about the asymptotic distribution of the (random) scalar quantity $$\alpha_n := n\mathbb E_{\hat{P}_n}[g(Z;\theta_n)]^T\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta_n)]^{-1}\mathbb E_{\hat{P}_n}[g(Z;\theta_n)]? $$ Here, $\mathbb E_{\hat{P}_n}[g(Z;\theta)] = (1/n)\sum_{i=1}^ng(z_i;\theta) \in \mathbb R^k$ is the empirical mean of the random vector $g(Z;\theta)$ from an i.i.d sample $z_1,\ldots,z_n\sim P$, and $\operatorname{Cov}_{\hat{P}_n}[g(Z;\theta)] \in \mathbb R^{k \times k}$ is the empirical covariance matrix.

Question 2: specialization to MLE

Same question with $g(Z;\theta) := -\partial_\theta \log p_\theta(Z)$, the score function for a parametric family of distributions (densities thereof) $\{p_\theta\}$, assumed to contain the true distribution of $Z$, namely $P$.

Question 3: specialization to location parameter estimation

Same question with $g(Z;\theta) = Z - \theta$.

Observations

My wild guesses are

  • $\alpha_n$ converges in distribution to some chi-squared distribution.
  • The solution of the problem will come from an application of the (functional) delta method.