I've seen in many introductory texts to matrices/linear algebra the claim "scalars are (just) $1$-by-$1$ matrices".

If this were true we could scalar-multiply only $1$-by-$n$ matrices...