This can be proved without introducing ultrafilters by name, by doing "finitary measure theory" and using Zorn's lemma.
An algebra $A$ on a set $X$ is just a $\sigma$-algebra without the $\sigma$, i.e. $\newcommand{\powerset}{\mathcal{P}}A \subseteq \powerset(X)$ and is closed under finite unions and complements (and therefore all other Boolean operations).
Let $\newcommand{\N}{\mathbb{N}}F \subseteq \powerset(\N)$ be the set of finite sets and their complements (so-called cofinite sets). This is an algebra. Furthermore, we can define a 2-valued finitely-additive measure $\mu : F \rightarrow \{0,1\}$ to be $0$ on the finite sets and $1$ on the cofinite sets. The existence of the required 2-valued finitely-additive measure on $\powerset(\N)$ then follows from:
Proposition For any algebra $A \subseteq \powerset(X)$ and finitely-additive 2-valued measure $\mu : A \rightarrow \{0,1\}$, there exists a finitely-additive measure $\overline{\mu} : \powerset(X) \rightarrow \{0,1\}$ extending $\mu$.
Proof: Most of the difficulty is in believing that it's true. We use Zorn's lemma. The poset consists of pairs $(B,\nu)$ where $B \supseteq A$ is an algebra of sets, and $\nu : B \rightarrow \{0,1\}$ is a finitely-additive measure extending $\mu$. The order relation $(B_1,\nu_1) \leq (B_2,\nu_2)$ is defined to hold when $B_1 \subseteq B_2$ and $\nu_2$ extends $\nu_1$. Every chain in this poset has an upper bound - we just take the union of algebras (this is the step that fails for $\sigma$-algebras) and define the measure on the union in the obvious way.
Let $(B,\nu)$ be a maximal element in the poset. Suppose for a contradiction that $B \neq \powerset(X)$, so there is some $U \in \powerset(X) \setminus B$. We contradict the maximality of $B$ by extending $\nu$ to a larger algebra $B'$ including $U$. Define $B' = \{ (U \cap S_1) \cup (\lnot U \cap S_2) \mid S_1, S_2 \in B \}$. It is clear that $B \subseteq B'$ and $U \in B'$, and with a little Boolean reasoning we can prove that for all $S_1,S_2,T_1,T_2 \in B$:
$$
((U \cap S_1) \cup (\lnot U \cap S_2)) \cup ((U \cap T_1) \cup (\lnot U \cap T_2))\\ = (U \cap (S_1 \cup T_1)) \cup (\lnot U \cap (S_2 \cup T_2))
$$
and
$$
\lnot ((U \cap S_1) \cup (\lnot U \cap S_2)) = (U \cap \lnot S_1) \cup (\lnot U \cap \lnot S_2)
$$
This proves that $B'$ is an algebra.
Now, define $d \in \{0,1\}$ to be the "outer measure" of $U$, i.e. $d = 0$ if there exists $S \in B$ such that $U \subseteq S$ and $\nu(S) = 0$, otherwise $d = 1$. Without loss of generality we can take $d = 1$, because we can exchange the roles of $U$ and $\lnot U$. We define $\nu'((U \cap S_1) \cup (\lnot U \cap S_2)) = \nu(S_1)$. This is well-defined because if $(U \cap S_1) \cup (\lnot U \cap S_2) = (U \cap T_1) \cup (\lnot U \cap S_2)$, then $U \cap S_1 = U \cap S_2$, so $U \subseteq \lnot (S_1 \triangle S_2)$, so as $d = 1$, $\nu(\lnot (S_1 \triangle S_2)) = 1$, and therefore $\nu(S_1) = \nu(S_2)$. The identities we used to prove that $B'$ is an algebra can then be used to prove that $\nu'$ is finitely additive, and it follows directly from the definition that it extends $\nu$. So we successfully contradicted the maximality of $B$. $\square$
Of course, I actually think ultrafilters are a good thing to know about, both in topology and logic. There is also no metamathematical benefit in doing it this way - over ZF the existence of a non-principal ultrafilter on $\N$ and a finitely-additive 2-valued measure on $\powerset(\N)$ are equivalent. The above proof is based on something I came up with while reproving Stone duality in the case where the points of the Stone space are defined to be Boolean homomorphisms into $2$, rather than ultrafilters. The proposition above is a special case of the fact that complete Boolean algebras (such as $2$) are injective objects in the category of Boolean algebras.