Let $p$ be the density of a continuous one-parameter exponential family distribution on $\mathbb{R}$. We assume that $$p(x) = c(x)\cdot \exp\bigl [ x \cdot \theta - b(\theta ) \bigr ], $$ where $\theta \in \mathbb{R}$ is the parameter. The score function is defined as $$ S(x) = \frac{\mathrm{d}( \log p(x))}{\mathrm{d}x} = \frac{ p'(x)}{p(x) } = \theta + c'(x) / c(x). $$ It can be shown that for $X \sim p$, $\mathbb{E} S(X) = 0$ and $\text{Var} [ S(X)] = I_p$, which is the Fisher's information of $p$.

Now we set $X_1, \ldots, X_n \sim p$ to be $n$ i.i.d. random variables. It is easy to see that $$ \frac{1}{n} \sum_{i=1}^n S(X_i) \xrightarrow{a.s} 0,~~\frac{1}{\sqrt{n} } \sum_{i=1}^n S(X_i)\xrightarrow{p} N(0, I_p). $$ I was wondering if there is any result on the concentration behavior of $\frac{1}{n} \sum_{i=1}^n S(X_i)$. Is it possible to obtain $$ \mathbb{P} \biggl [ \frac{1}{n} \sum_{i=1}^n S(X_i) > t \biggr ] \leq \exp( - n \cdot C \cdot t^{\alpha} ) $$ for some constants $C>0$ and $\alpha >0$? In particular, if $\alpha = 2$, we just show that $\frac{1}{n} \sum_{i=1}^n S(X_i)$ has a Gaussian-type tail.

  • $\begingroup$ Do you want $\frac {\partial }{\partial \theta}$ in the score function or what you have ? $\endgroup$ – user83457 Jan 20 '17 at 13:09
  • $\begingroup$ @michael the score function does involves $\partial / \partial \theta$. For instance, if $p$ is the density of $N(0,1)$ distribution, then the score function is $s(x) = x$. $\endgroup$ – Steve Jan 20 '17 at 23:12

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.