Concentration inequalities specialized for log-likelihood / log-density functions

Question

Let $P$ be a probability measure and $f$ be some probability density function (not necessarily related to $P$). Consider the function $$ L(X_1,\ldots,X_n) =\frac1n\sum_{i=1}^n\log f(X_i), \quad X_i\overset{iid}{\sim}P. $$

This is just the empirical log-likelihood from statistics. Are there any concentration inequalities that are specialized for $L(X_1,\ldots,X_n)$?

Surprisingly, I could not find any in the literature besides the generic ones. That is, we can naively apply generic inequalities such as Bernstein, etc. or assume that $\log f(X_i)$ is a subgaussian random variable, but this is a bit unsatisfying. For example, in the simplest case where $f$ is a Gaussian density, $\log f(x)$ is a quadratic function, and hence not subgaussian if $X_i$ is also Gaussian.

It seems like it should be possible to exploit the fact that $f$ is a density to come up with something more useful.

Answer 1

Assume $X$ has density $f$ and put $Y=\log f(X)$. Then, to compute the moment-generating function of $Y$, we write $$ E e^{\lambda Y} =E e^{\lambda\log f(X)} =E[f(X)^{\lambda}] =\int_{-\infty}^\infty f^{\lambda+1}(x)dx. $$ For $\lambda\ge0$, the latter integral is finite for all densities, because the region where $f\ge1$ must have (Lebesgue) measure $\le 1$, and elsewhere $f<1$ so raising it to a power $>1$ only decreases the value. So the answer depends on your ability to analytically control $F(\lambda):=\int_{-\infty}^\infty f^{\lambda+1}(x)dx$. In the normal Gaussian case, we have $F(\lambda)=1/\sqrt{\lambda+1}$. You can then apply the standard exponential bounding technique (plus Markov's inequality) to the sum of $Y$'s.