Qiaochu's answer captures very nicely the ordinary categorical aspects of the category $\mathbf{Vect}_k$ of finite-dimensional vector spaces over $k$; I'll just add some homotopy-theoretic comments that I find interesting.

I am fond of the fact that we can think of vector bundles in the following way: for a topological field $k$, the category $\mathbf{Vect}_k$ is closed monoidal, and thus also enriched over $\mathcal{T}op$, the category of topological spaces. We can thus consider $\mathbf{Vect}_k$ as an $(\infty,1)$-category, and vector bundles over a space $X$ correspond to functors $X\to\mathbf{Vect}_k$; this is exactly analogous to how Grothendieck opfibrations $E\to B$ correspond to pseudofunctors $B\to \mathbf{Cat}$ (and coCartesian fibrations of quasicategories generalize both these examples).

To put things in perspective, this is really what's going on when we consider the classifying spaces $BO(n)$ or $BU(n)$ (recall that homotopy classes of maps $X\to BO(n)$ classify $n$-dimensional real vector bundles over $X$ and similarly with $BU(n)$ for complex vector bundles). Recall that spaces are $\infty$-groupoids. If $X$ is connected, then it's an $\infty$-groupoid with only one object (up to equivalence), so a functor $X\to\mathbf{Vect}_k$ is defined on objects just by choosing a single object $V\in\mathbf{Vect}_k$, and we can consider functors $X\to\mathcal{V}$, where $\mathcal{V}$ is the full sub-$\mathcal{T}op$-category of $\mathbf{Vect}_k$ with $V$ as its only object. And because $X$ is an $\infty$-groupoid, any such functor factors through $\mathrm{Core}(\mathcal{V})$, where $\mathrm{Core}(\mathcal{V})$ is the full subcategory of $\mathcal{V}$ of automorphisms of $V$. When we go back to thinking of the $\infty$-groupoid $\mathrm{Core}(\mathcal{V})$ as a space, we wind up with $BO(n)$ or $BU(n)$ (for real and complex vector spaces of dimension $n$ respectively).