Note from OP: I gave up and reposted this Question with a Bounty on Cross Validated HERE.
In certain applications, we approximate an unknown pdf by placing uniformly weighted Gaussian terms at each of some sample points $\{x_{1},...,x_{n}\} $ and assigning some variances $\{\sigma_{1},...,\sigma_{n}\} $:
$$f(x)\equiv\frac{1}{n}\sum_{i=1}^{n}N(x-x_{i},\sigma_{i})=\frac{1}{n}\sum_{i=1}^{n}\frac{1}{\sqrt{2\pi\sigma_{i}^{2}}}e^{-\frac{(x-x_{i})^{2}}{2\sigma_{i}^{2}}}$$
It seems intuitive that the sampled log-liklihood would be greater than (or equal to) the mean log-liklihood:
$$\frac{1}{n}\sum_{j=1}^{n}ln(f(x_{j}))\geq\int f(x)ln(f(x))dx $$
This is obviously true for small variances (each $x_{i}$ is on top of a narrow Gaussian) and for very large variances (all $x_{i}$'s are together atop one broad Gaussian), and it's true for every random set of $x_i$'s and $\sigma_i$'s variances I've tried, but I can't prove it's true for all sets of variances $\{\sigma_{1},...,\sigma_{n}\}$. I suspect Jensen's Inequality would be part of such a proof, but I'm stumped.
