To my mind there are two classes of interesting categorical facts here, loosely speaking "additive" facts and "multiplicative" facts. Some additive facts:
Finite-dimensional vector spaces over $k$ has biproducts, and every object is a finite biproduct of copies of a single object, namely $k$. The categories with this property are precisely the categories of finite rank free modules over a semiring $R$ (the endomorphisms of the single object). These biproduct decompositions encapsulate both the idea that vector spaces have bases and that a choice of bases can be used to write linear maps as matrices.
The single object $k$ above is simple, and so every object is a finite biproduct of simple objects. The categories with this property, in addition to 1, are precisely the categories of finite rank free modules over a division semiring $R$.
Note that additive facts can't see anything about fields being commutative. The multiplicative facts can:
Finite-dimensional vector spaces over $k$ is symmetric monoidal with respect to tensor product, and is also closed monoidal and has duals (sometimes called compact closed). This observation encapsulates the yoga surrounding tensors of various types (e.g. endomorphisms $V \to V$ correspond to elements of $V \otimes V^{\ast}$), as well as the existence and basic properties (e.g. cyclic symmetry) of the trace.
The single object $k$ above is the tensor unit, and so every object is a finite biproduct of copies of the unit. Also, the monoidal structure is additive in each variable. I believe, but haven't carefully checked, that the (symmetric monoidal) categories with this property, in addition to 1 and 2 above, are precisely the (symmetric monoidal) categories of finite rank free modules over a commutative division semiring ("semifield") $R$. This encapsulates the concrete description of tensor products as a functor in terms of Kronecker products.
There's surprisingly little to say as far as fields being rings as opposed to semirings, though. This mostly becomes relevant when we reduce computing (co)equalizers to computing (co)kernels by subtracting morphisms, as in any abelian category.
Edit: I haven't mentioned the determinant yet. This mixes additive and multiplicative: abstractly the point is that we have a natural graded isomorphism
$$\wedge^{\bullet}(V \oplus W) \cong \wedge^{\bullet}(V) \otimes \wedge^{\bullet}(W)$$
where $\wedge^{\bullet}$ denotes the exterior algebra (which we need the symmetric monoidal structure, together with the existence of certain colimits, to describe). It follows that if $L_i$ are objects which have the property that $\wedge^k(L_i) = 0$ for $k \ge 2$ then
$$\wedge^n(L_1 \oplus \dots \oplus L_n) = L_1 \otimes \dots \otimes L_n.$$
Combined with the facts above this gives the existence and basic properties of the determinant, more or less. Note that the exterior algebra can be defined by a universal property, but to verify that the standard construction has this universal property we need the symmetric monoidal structure to distribute over finite colimits. Fortunately this is implied by compact closure.