Question: Is the center of the automorphism group of a von Neumann algebra $\mathscr{M}$ trivial (=$\{\mathrm{id}\}$) whenever $\mathscr{M}$ is a factor (=$\mathscr{M}$ has center $\{\lambda I; \lambda \in \mathbb{C}\}$)?
It is true in the finite dimensional case. A finite dimensional factor is $M_n(\mathbb{C})$ for some $n \in \mathbb{N}$.
We proceed in two steps. First, every automorphism of $M_n$ is of the form $a \mapsto u^*a u$ for some unitary $u$. We will show that if an automorphism is in the center of $\mathop{\mathrm{Aut}}$, then $u$ is in the center of its unitary group. Finally, we show that the center of the unitary group is "trivial".
There is a surjective group homomorpism $\varphi\colon U(n) \to \mathop{\mathrm{Aut}}(M_n)$ with $\varphi(u)(a)=u^*au$. The kernel of $\varphi$ are the unitaries in the center of $M_n$. Thus $\ker \varphi = \{\lambda I;\ |\lambda|=1\}$. Now, given any $u$ such that $\varphi(u)$ is in the center of $\mathop{\mathrm{Aut}}(M_n)$. Then for any other $v$ we have $vuv^{-1}u^{-1} \in \ker\varphi$. Thus $vuv^{-1}u^{-1}=\lambda I$ for some $\lambda \in \mathbb{C}$ with $|\lambda|=1$. In particular, when $u=v$ we find $I=\lambda I$. Apparently $vuv^{-1}u^{-1}=I$. Thus $u$ is in the center of $U(n)$.
The embedding $e\colon U(n) \to GL(n)$ is an irreducible group representation. A unitary $u$ in the center of $U(n)$, is an intertwining map from this representation to itself. Thus, by Schur's lemma, it is $\lambda I$ for some $\lambda \in \mathbb{C}$. Clearly $|\lambda| = 1$. Thus the center of $U(n)$ is the kernel of $\varphi$.