The solution is in the comment of Asaf Karagila, which points to the paper "A geometric form of the axiom of choice" by Bell and Fremlin.
In fact, this paper asserts that the assertion : "There exists a normed space such that the unit ball of its dual contains extreme points" is equivalent to the axiom of choice. So it doesn't matter how nice the space $X$, we can't prove the Krein-Miman for $M(X)=C(X)'$ in ZF. As a corollary, the ultrafilter lemma and the Krein Milman theorem imply (so is equivalent to) the axiom of choice, so the Krein Milman theorem cannot be provable only with the ultrafilter lemma, as the ultrafilter lemma is not equivalent to the axiom of choice. In particular, it shows that we can't prove the Krein Milman theorem without at least a weak form of axiom of choice, and as the ultrafilter is not enough, maybe we really need the axiom of choice in full generality, but the paper doesn't answer this question.