divide and conquer

$f$ is over $n$ vars jointly. The other case is already a single-letter problem whose answer can (in principle) be numerically estimated.

My current thinking is to find $\min_s n^{-1}H(y^n|x^n \in s)$ over sets $s$ of $\exp(n(H(x)-R))$ sequences typical for $x^n$, taking $n\to\infty$. Then if one can design $f$ with fibers similar to the optimum $s$, the main question is solved. I would be surprised if this new min wasn't equal to the information bottleneck optimum, which would make design of $f$ straightforward.

This problem arises immediately in MIMO radar direction estimation. See, for example, MUSIC

Why do you want to study information geometry?

$H$ is (conditional) Shannon entropy. $(x,y)$ is distributed over $\mathcal{X}\times \mathcal{Y}$, both sets finite.