For finite matrices, your norm is the spectral radius of $A$. Indeed, for each matrix $A$ you can construct a Jordan-like form $A=VJV^{-1}$ in which the strictly upper diagonal part contains $\varepsilon$ instead of 1; then define a norm $\|M\|:=\|V^{-1}MV\|_2$ (where $\|K\|_2$ is the operator norm of $K$), and then $\|A\| = \rho(A) + O(\varepsilon)$. On the other hand, each norm is greater than the spectral radius.