Here is a non-trivial condition:
If $x$ belongs to this intersection, then $x$ commutes with every nilpotent element.
Proof. For every invertible $a\in A$, the function
$$N(z):=\lVert a^{-1}za\rVert$$
is a norm of unital algebra. Let $n$ be a nilpotent element, of order $k$, and choose $a_t=\mathbf1-tn$ in the construction above, with $t\in\mathbb R$ a parameter. Then
$$a_t^{-1}xa_t=(\mathbf 1+tn+\dotsb+t^{k-1}n^{k-1})x(\mathbf1-n)=x+t(nx-xn)+\cdots-t^kn^{k-1}xn$$
is a polynomial function of $t$.
By assumption, $t\mapsto\lVert a_t^{-1}xa_t\rVert$ is a bounded (by $1$) function, hence the polynomial above needs to be constant. In particular $nx-xn=0$.
As a corollary, the answer for the case of $A=M_r({\mathbb C})$ with $r\ge2$ is as you expected. Write $X$ instead of $x$ (it is a matrix). Let $u\in{\mathbb C}^r$ be a non-zero vector. Choose $v$ such that $v^Tu=0$ (it exists). Then $uv^T$ is nilpotent, hence $Xuv^T=uv^TX$, which implies that $Xu$ is parallel to $u$. Thus every vector is an eigenvector, which implies that $X$ is scalar: $X=\alpha I_r$.
Addition. Suppose now that $(A,\|\cdot\|)$ is a unital Banach algebra. The spectral radius
$$r(u)=\lim\inf\|u^k\|^{1/k}$$
is well-defined. As above, $\cal N$ contains all norms $N_a=\|a^{-1}\cdot a\|$ for $a\in A^\times$. If $x$ belongs to the OP's intersection, then the set
$$\{a^{-1}xa\;|a\in A^\times\}$$
is bounded. Consider an element $u\in A$ for which $r(u)=0$ ($u$ can be be nilpotent, but this is not necessary if $A$ is infinite dimensional). Then ${\bf1}-zu$ is invertible for every $z\in\mathbb C$. Thus
$$z\mapsto({\bf1}-zu)^{-1}x({\bf1}-zu)$$
is a bounded entire function, hence a constant function, $\equiv x$. In other words $x({\bf1}-zu)\equiv ({\bf1}-zu)x$, that is $xu=ux$. Thus the property mentionned above extends to:
If $x$ belongs to this intersection, then $x$ commutes with every element of spectral radius $0$.
\lVert\rVert
rather than\Vert\Vert
and$\mathbf1$
rather than${\bf1}$
are as it were best practices. I hope it hardly changed the rendering. You mentioned spacing around the dots, which did indeed change, but can be restored by\lVert\,\cdot\,\rVert
in a way that doesn't incur the other issues of\Vert
. But, @AliBagheri, I apologise if it was unwelcome. $\endgroup$