I believe that the answer is yes. Put $c:=g(x_1,\ldots, x_n)$, which is irreducible in the UFD $R:=k[x_1,\ldots, x_n]$. Assume $f$ is irreducible, but also assume by way of contradiction that $f(c)=\alpha\beta$ in $R$, with $\alpha,\beta\notin k$. Write $\alpha=\sum \alpha_i$ and $\beta=\sum \beta_j$, where the $\alpha_i$ and $\beta_j$ are homogeneous (of the appropriate degrees). [Note: We implicitly only consider terms with nonzero support in the remainder.]
We can write $\alpha_i=c^{e_i}\alpha_i'$ with $e_i$ maximal. Similarly write $\beta_j=c^{e_j'}\beta_j'$. Fix $m_1$ maximal with $\deg(\alpha_{m_1}')$ maximal among $\{\deg(\alpha_i')\}$. Similarly, fix $m_2$ maximal with $\deg(\beta_{m_2}')$ maximized.
Case 1: $\deg(\alpha_{m_1}')=0$ and $\deg(\beta_{m_2}')=0$. In this case $\alpha$ and $\beta$ are polynomials in $c$, which contradicts the irreducibility of $f$.
Case 2: Without loss of generality, $\deg(\alpha_{m_1}')>0$.
Consider the degree $m_1+m_2$ coefficient of $f(c)=\alpha\beta$. On the one hand, since $f(c)$ is a polynomial in $c$, it is either zero or a $k$-multiple of a power of $c$, say $c^{e}$. On the other hand, this term on the right-hand side is
$$c^{e_{m_1}+e_{m_2}'}\alpha_{m_1}'\beta_{m_2}' + \sum_{(i,j)\neq (m_1,m_2)\ :\ i+j=m_1+m_2}\alpha_i\beta_j.$$
Each term in the big sum is divisible by strictly more powers of $c$ than $e_{m_1}+e_{m_2}'$, by maximality of degrees. Also $e>e_{m_1}+e_{m_2}'$, since $\deg(\alpha_{m_1}')>0$. This gives us the necessary contradiction.