This is just a sketch, you may refer to Miles Reid's "Decomposition of toric morphisms" for details.
The answer is yes for surfaces (see Lemma 10.4.1 in Cox, Little, Schenck, for instance).
I would say that this is also true if $\dim X_{\Sigma'}\geq 3$.
Let us consider a wall $\omega_n\in \Sigma'(n-1)$ generated by some primitive vectors $u_1,\ldots,u_{n-1}$ in the lattice $N$. Since $X_{\Sigma'}$ is smooth all the cones will be simplicial. In particular, $\omega_n$ separates only two (smooth) cones of maximal dimension
$$ \sigma_n=\operatorname{cone}(u_1,\ldots,u_{n-1},u_n)$$
and
$$ \sigma_{n+1}=\operatorname{cone}(u_1,\ldots,u_{n-1},u_{n+1}).$$
Thus, there are integers $b_1,\ldots,b_{n+1}\in \mathbb{Z}$ and a "wall relation"
$$b_nu_n + \sum_{i=1}^{n-1}b_i u_i + b_{n+1}u_{n+1}=0 $$
(Here we have that $b_n$ and $b_{n+1}$ are positive, by convexity)
Let $\alpha = \#\{ i\;|\; b_i<0\}$ and $\beta = n - \#\{ i\;|\;b_i>0 \}$.
By the Contraction Theorem in Reid's article, there exists a contraction $\varphi_R:X_{\Sigma'}\to X_R$ in the sence of Mori theory contracting all curves whose numerical class belong to the ray $R=\mathbb{R}_{\geq 0}[V(\omega_n)]$, such that $\dim \operatorname{Exc}(\varphi_R)=n-\alpha$ and that $\dim\varphi_R(\operatorname{Exc}(\varphi_R))=\beta$.
If we look at the special case where $u_2,\ldots,u_n,u_{n+1}$ are the elements of the standard basis of $\mathbb{Z}^n$ (say $u_i=e_{i-1}$) and the morphism $X_{\Sigma'}\to X_\Sigma$ is given by the contraction that sends the divisor $V(u_1)$ to a point, then we will have that
$$ b_1u_1 = -\sum_{i=2}^{n+1}b_iu_i. $$
In coordinates, $b_1u_1=(-b_2,\ldots,-b_{n+1})$ and thus we can suppose $b_1=-1$, dividing the wall relation by $-b_1>0$, if necessary.
Now, it should be noticed (by computing determinants, for example) that both cones $\sigma$ and $\sigma'$ are smooth if and only if $b_n=b_{n+1}=1$.
Now, the exceptional divisor $V(u_1)$ has exactly $n$ invariant points that corresponds precisely to the cones of maximal dimension
$$ \sigma_i = \operatorname{cone}(u_1,u_2,\ldots, \widehat{u_i},\ldots,u_{n+1}),$$
for $i=2,\ldots,n+1$, that contains the ray generated by $u_1$. By considering the walls $\omega_i=\operatorname{cone}(u_1,\ldots,\widehat{u_i},\ldots,u_{n})$, for $i=2,\ldots,n$, that separates $\sigma_{n+1}$ and $\sigma_i$ and that define the same contraction, we obtain by the same computations that $u_1=(1,\ldots,1)$ is the barycenter.
I think that with the same kind of arguments you can get a positive answer when the contraction sends the divisor onto a subvariety of higher dimension.