There’s a recent book by Meckes and Meckes (published by Cambridge) that might work. Vector spaces appear early, as do linear transformations and eigenvalues/vectors. Determinants are delayed but can be moved earlier if desired. I am using it for the first time and students seem to find it difficult but not impossible.