The result on affinity of the entropy map holds for arbitrary finite convex combinations of invariant measures: if $\mu_1,\dots,\mu_n\in M(X,T)$ are any invariant measures and $a_1,\dots, a_n\in [0,1]$ sum to 1, then $h(\sum a_i \mu_i) = \sum a_i h(\mu_i)$, where I write $h(\mu)$ in place of $h_\mu(T)$.
This can be proved by induction in a pretty standard way. The case $n=2$ is proved already. Now if it holds for some $n$, we prove it for $n+1$: first note that $\sum_{i=1}^n a_i \mu_i = (1-a_{n+1}) \nu$, where $\nu\in M(X,T)$ is a probability measure, and we have
$$
h(\sum_{i=1}^{n+1} a_i \mu_i) = h((1-a_{n+1})\nu + a_{n+1}\mu_{n+1}) = (1-a_{n+1})h(\nu) + a_{n+1} h(\mu_{n+1}),
$$
and then expanding $h(\nu)$ using the induction hypothesis gives the result we want.
The remark after Theorem 8.1 in Walters' book doesn't say quite what you suggested it does. Quoting the remark: "The first part of the proof shows that if $\mu,m\in M(X)$, $p\in [0,1]$, and $\xi$ is a finite partition then $H_{p\mu + (1-p)m}(\xi) \geq p H_\mu(\xi) + (1-p)H_m(\xi)$." Indeed this only gives an inequality in one direction, but two points must be made:
- The measures $\mu,m$ are not assumed to be invariant.
- The quantity here is $H_\mu(\xi)$, the entropy of a partition, rather than $h_\mu(T)$, the (Kolmogorov-Sinai) entropy of an invariant measure, which is defined as the (linear) growth rate of the quantities $H_\mu(\xi_n)$ for a particular sequence of partitions $\xi_n$. So the remark is not about the KS entropy $h_\mu(T)$, which is affine over all finite convex combinations, but rather about a specific intermediary quantity which is not generally affine, although it does satisfy a super-affinity inequality. In particular there is no contradiction between the remark and the fact that affinity of $h_\mu$ holds for arbitrary finite combinations.