Consider an optimization problem as follows: $$ \min\mathbb E_w[f_0(w)] \mathrm{\,\,\,\,\,\ s.t.\,\,\,\,} E_w[f_i(w)]\leq 0 ,\,\,\, i=1,\dots, k $$ where the maximum is taken over $\mathscr M$, the space of probability measureson $(X,\mathscr B)$. Indeed the problem can be written as : $$ \min_{\mu\in\mathscr M} \int_Xf_0\mathrm{d}\mu \mathrm{\,\,\,\,\,\ s.t.\,\,\,\,} \int_Xf_i\mathrm d\mu\leq 0 ,\,\,\, i=1,\dots, k. (\star) $$ This problem occurs particularly in information theory where the goal is to maximize certain entropy function or mutual information subject to some assumptions on the random variable variance.
Ex.1. Differential entropy of a continuous random variable $Y$ with the variance bounded by $\sigma^2$ is $\frac 12\log(2\pi e \sigma^2)$ and it is obtained by choosing $Y$ as Gaussian. This can be proved using positivity of KL-divergence $D(f||g)$ and choosing $g$ Gaussian distribution.
Similar statements can be found in different entropy maximization problems with similar proof techniques. However, the previous example is rather an exception and the problem can easily get more complicated, for instance when the difference of two mutual information should be maximized.
Apart from information theoretic context, the problem $(\star)$ is independently interesting.
Here is the question: Has there been any work on solving $(*)$ in a systematic way, something like classical optimization theory? Are there any special cases of the problem, like those mentioned above, for which the problem can be solved?
Remark 1 I can spot a similarity with Kantorovich's formulation of optimal transportation problem.
Update 1 From the comments below, a solution can be found when $f_i$'s are not changing with the choice of $\mu$. An answer can be found here: semi-infinite linear programming. In this case, the objective function and constraints are linear functional of $\mu$ and therefore one can apply similar methods from linear programming with proper considerations. The solution to this problem is a finite measure with $k+1$ points.
