I'm trying to teach myself category theory from Steve Awodey's Category Theory. Chapter 2 asserts:

It is not hard to see that a filter F is an ultrafilter just if for every element b ∈ B, either b ∈ F or ¬b ∈ F, and not both (exercise!).

I've managed to prove the backwards implication, but the forwards implication is eluding me. I'm guessing the correct approach is to consider a filter F such that there exists a b ∈ B such that neither b ∈ F or ¬b ∈ F, and construct a superset filter F' which contains b, but I can't figure out how to construct F' and prove that it's a filter. Any hints much appreciated!