For infinite measures $\mu$, the pushforward measure need not be outer regular,
even if $\Sigma$ is the $\sigma$-algebra of Borel sets.
For a simple counterexample, let $A$ be the real line $\mathbb{R}$,
and let $B$ be the rationals $\mathbb{Q}$ (with subspace topology).
Let $\mu$ be Lebesgue measure. Write the rationals using an injective
integer-indexed enumeration $(q_z)_{z \in \mathbb{Z}}$.
Let $X$ be the function from $\mathbb{R}$ to $\mathbb{Q}$ defined
by setting $X^{-1}(q_z)$ to be the half-open interval $[z,z+1)$.
The inverse image of any subset of $\mathbb{Q}$ under $X$ is a countable
union of half-open intervals, hence Borel. Thus the function $X$ is (Borel) measurable.
But the pushforward measure $X_{\star}(\mu)$ is the counting (cardinality) measure on
$\mathbb{Q}$. That is $X_{\star}(\mu)(Z) = |Z|$ for any subset $Z \subseteq \mathbb{Q}$.
This is not outer regular because the measure of a nonempty open is always $\infty$.
The case of a finite measure $\mu$ has a trivially affirmative answer in the
special case that the topological space $B$ is regular, because any finite measure
$\nu$ on a second-countable regular space $B$ is automatically outer regular.
(I'm sorry I don't know a direct reference this, though it must be standard. I
have pieced it together from the following. By second-countability the
restriction of $\nu$ to open sets is a continuous valuation. Any finite continuous valuation
on a regular space extends, via outer measure defined using opens, to an outer-regular
measure $\nu'$ on Borel sets. But $\nu$ and $\nu'$ are two finite Borel measures
agreeing on open sets, and hence equal. Thus $\nu$ is outer regular. For the extension
part of this argument see Theorem 4.4 of Alvarez-Manilla, "Extension of valuations
on locally compact topological spaces", Topology and its Applications,
124(3):397-433 (2002).)
I do not know the answer for finite $\mu$ and non-regular $B$.
