Let $\boldsymbol{x} = (\boldsymbol{x}_1, \dots, \boldsymbol{x}_n)$ be a $n$-dimensional *random* vector on $\mathbb{R}$ (i.e. $\boldsymbol{x}$ is a random variable). Suppose we have a binary function
$f: \mathbb{R}^n \to \{0,1\}$ such that $\mathrm{H}(f(\boldsymbol{x})) = \log 2 = 1$, where $\mathrm{H}$ is the entropy function.
Consider the lower bound of
$$
S_k = \sum_{\substack{A \subseteq \{1,\dots,n\} \\ |A|=k}}\mathrm{I}((\boldsymbol{x}_i : i \in A);f(\boldsymbol{x}))
$$
where $\mathrm{I}$ stands for mutual information.
It is easy to see that when $k<n$, the lower bound is exactly $0$ since we could construct $\boldsymbol{x}$ and $f$ like

- $\boldsymbol{x}$ satisfies that $\boldsymbol{x}_1,\dots,\boldsymbol{x}_n$ are i.i.d. and $\boldsymbol{x}_1$ has the $1/2$ probability to be $0$ and $1$, respectively.
- $f((x_1,\dots,x_n)) = (x_1 + \cdots + x_n) \bmod{2}$.

However, if we restrict $\boldsymbol{x}$ to follow $$ \sum_{i=1}^n \Pr(\boldsymbol{x}_i \neq 0) \leq \epsilon n $$ with some "small" $\epsilon \in [0,1]$. It seems like that we could get a non-trivial lower bound.

My question:

Can we derive a non-trivial lower bound of $S_k$ with given $n$ and $\epsilon$?

Any idea or reference would be welcome indeed.

**Background.** This question is an abstract from a point-to-point communication with $n$ channels observed by a partial eavesdropper. "$0$" means no message. Hence the last restriction above actually controls the communication complexity, to some extent.

**Trivial bound.** Finding the bound of $S_k$ is equivalent to study
$$
T_k = \frac{1}{\binom{n}{k}}\sum_{\substack{A \subseteq \{1,\dots,n\} \\ |A|=k}}\mathrm{H}(f(\boldsymbol{x}) | (\boldsymbol{x}_i : i \in A))
$$
By some trivial method it seems that
$$
T_k \leq \mathrm{H}(\epsilon(n-k))
$$
if $\epsilon(n-k) \leq 1/2$, where $\mathrm{H}(p) = - p \log p - (1-p) \log (1-p)$. I think it is quite loose.

Cross-posted at https://cstheory.stackexchange.com/q/39053.

notbe binary. $\endgroup$ – Lwins Sep 10 '17 at 7:25