This is just an amplification of Emerton and Pietro Majer's answers, but got a bit too big.

You can give a geometric interpretation of all not necessarily unital (NNU) $k$-algebras, for any ring k. The category of NNU $k$-algebras is equivalent to the category of augmented (unital) $k$-algebras, by which I mean $k$-algebras $A$ with a retraction $A\to k$ of the structure map, where the morphisms are the $k$-algebra maps that commute with the retractions. An augmented $k$-algebra $A$ corresponds to the NNU $k$-algebra which is the kernel of its augmentation map $A\to k$. Thus the category of NNU $k$-algebras is anti-equivalent to the category of pointed affine $k$-schemes, where the maps preserve the pointed structure. The forgetful functor from $k$-algebras to NNU $k$-algebras then has a geometric interpretation. It corresponds to the functor from affine $k$-schemes to pointed affine $k$-schemes that sends $X$ to the disjoint union of $X$ and $\mathrm{Spec}(k)$, where the second component is the distinguished point.

In particular, an NNU map $A\to B$ of $k$-algebras corresponds to a scheme map 
$$\mathrm{Spec}(B)\coprod \mathrm{Spec}(k) \to \mathrm{Spec}(A)\coprod \mathrm{Spec}(k)$$
which is the identity on the second component. The original map is unital if and only if the map takes the $\mathrm{Spec}(B)$ component to the $\mathrm{Spec}(A)$ component. But in general there could be a connected component of $\mathrm{Spec}(B)$ taken to the $\mathrm{Spec}(k)$ component. These are exactly the vaporized components in Emerton's answer.

Here's a little exercise. What does the NNU subring $n\mathbf{Z}$ of $\mathbf{Z}$ look like geometrically?

So the question about whether it's better to look at unital or NNU rings is (at least in the commutative case) the same as the question of whether it's better to looked a pointed or unpointed spaces, which also comes up in homotopy theory. I prefer the unital/unpointed approach (no doubt because of my education), but it's easy to translate back and forth between the two.