Let $p$ be the density of a continuous one-parameter exponential family distribution on $\mathbb{R}$. We assume that $$p(x) = c(x)\cdot \exp\bigl [ x \cdot \theta - b(\theta ) \bigr ], $$ where $\theta \in \mathbb{R}$ is the parameter. The score function is defined as $$ S(x) = \frac{\mathrm{d}( \log p(x))}{\mathrm{d}x} = \frac{ p'(x)}{p(x) } = \theta + c'(x) / c(x). $$ It can be shown that for $X \sim p$, $\mathbb{E} S(X) = 0$ and $\text{Var} [ S(X)] = I_p$, which is the Fisher's information of $p$.

Now we set $X_1, \ldots, X_n \sim p$ to be $n$ i.i.d. random variables. It is easy to see that $$ \frac{1}{n} \sum_{i=1}^n S(X_i) \xrightarrow{a.s} 0,~~\frac{1}{\sqrt{n} } \sum_{i=1}^n S(X_i)\xrightarrow{p} N(0, I_p). $$ I was wondering if there is any result on the concentration behavior of $\frac{1}{n} \sum_{i=1}^n S(X_i)$. Is it possible to obtain $$ \mathbb{P} \biggl [ \frac{1}{n} \sum_{i=1}^n S(X_i) > t \biggr ] \leq \exp( - n \cdot C \cdot t^{\alpha} ) $$ for some constants $C>0$ and $\alpha >0$? In particular, if $\alpha = 2$, we just show that $\frac{1}{n} \sum_{i=1}^n S(X_i)$ has a Gaussian-type tail.