I believe the following is a counterexample which can be defined for arbitrarily large $p$'s; please correct me if I'm mistaken.

Consider $U=\left\{ \begin{pmatrix}1&a&b\\&1&a^p\\&&1\end{pmatrix}:a,b\in k\right\}\subseteq\mathrm{GL}_3(k)$ and take $u=u_\lambda=\begin{pmatrix}1&\lambda\\&1&\lambda\\&&1\end{pmatrix}$ with $\lambda\in\mathbb{F}_p$ (i.e. $\lambda=\lambda^p$). Then one easily computes that $$\begin{pmatrix}1&a&b\\&1&a^p\\&&1\end{pmatrix}\in Z_U(u)\quad\iff\quad a=a^p, $$ and thus $Z_U(u)\simeq \mu_p\ltimes \mathbb G_a$ and $Z_U(u)^\circ=\begin{pmatrix}1&&*\\&1&\\&&1\end{pmatrix}\simeq \mathbb G_a$ (here $\mu_p$ is the group of $p$-th roots of $1$). In particular, if we take $\lambda\ne 0$ then $u\notin Z_U(u)^\circ$.

The following is a counterexample which can be defined for arbitrarily large $p$'s.

Consider $U=\left\{ \begin{pmatrix}1&a&b\\&1&a^p\\&&1\end{pmatrix}:a,b\in k\right\}\subseteq\mathrm{GL}_3(k)$ and take $u=u_\lambda=\begin{pmatrix}1&\lambda\\&1&\lambda\\&&1\end{pmatrix}$ with $0\ne \lambda\in\mathbb{F}_p$ (i.e. $\lambda=\lambda^p$). Then one easily computes that $$\begin{pmatrix}1&a&b\\&1&a^p\\&&1\end{pmatrix}\in Z_U(u)\quad\iff\quad a=a^p, $$ and thus $Z_U(u)\simeq \mu_p\ltimes \mathbb G_a$ and $Z_U(u)^\circ=\begin{pmatrix}1&&*\\&1&\\&&1\end{pmatrix}\simeq \mathbb G_a$ (here $\mu_p$ is the group of $p$-th roots of $1$). In particular  $u\notin Z_U(u)^\circ$.

I believe the following is counterexample which can be defined for arbitrarily large $p$'s; please correct me if I'm mistaken.

Consider $U=\left\{ \begin{pmatrix}1&a&b\\&1&a^p\\&&1\end{pmatrix}:a,b\in k\right\}\subseteq\mathrm{GL}_3(k)$ and take $u=u_\lambda=\begin{pmatrix}1&\lambda\\&1&\lambda\\&&1\end{pmatrix}$ with $\lambda\in\mathbb{F}_p$ (i.e. $\lambda=\lambda^p$). Then one easily computes that $$\begin{pmatrix}1&a&b\\&1&a^p\\&&1\end{pmatrix}\in Z_U(u)\quad\iff\quad a=a^p, $$ and thus $Z_U(u)\simeq \mu_p\ltimes \mathbb G_a$ and $Z_U(u)^\circ=\begin{pmatrix}1&&*\\&1&\\&&1\end{pmatrix}\simeq \mathbb G_a$ (here $\mu_p$ is the group of $p$-th roots of $1$). In particular, if we take $\lambda\ne 0$ then $u\notin Z_U(u)^\circ$.

I believe the following is a counterexample which can be defined for arbitrarily large $p$'s; please correct me if I'm mistaken.

Consider $U=\left\{ \begin{pmatrix}1&a&b\\&1&a^p\\&&1\end{pmatrix}:a,b\in k\right\}\subseteq\mathrm{GL}_3(k)$ and take $u=u_\lambda=\begin{pmatrix}1&\lambda\\&1&\lambda\\&&1\end{pmatrix}$ with $\lambda\in\mathbb{F}_p$ (i.e. $\lambda=\lambda^p$). Then one easily computes that $$\begin{pmatrix}1&a&b\\&1&a^p\\&&1\end{pmatrix}\in Z_U(u)\quad\iff\quad a=a^p, $$ and thus $Z_U(u)\simeq \mu_p\ltimes \mathbb G_a$ and $Z_U(u)^\circ=\begin{pmatrix}1&&*\\&1&\\&&1\end{pmatrix}\simeq \mathbb G_a$ (here $\mu_p$ is the group of $p$-th roots of $1$). In particular, if we take $\lambda\ne 0$ then $u\notin Z_U(u)^\circ$.