You could view a biquadratic field $\mathbf Q(\sqrt{a},\sqrt{b})$ as the top of a tower of two quadratic extensions: it is quadratic over $\mathbf Q(\sqrt{b})$, which in turn is quadratic over $\mathbf Q$. Then show for a quadratic extension of number fields $E'/E$ that
$$
\zeta_{E'}(s) = \zeta_{E}(s)L(s,\chi),
$$
where $\chi$ is the (unique) nontrivial character of ${\rm Gal}(E'/E)$ and $\chi(\mathfrak p)$ in the Euler factor $1/(1 - \chi(\mathfrak p)/{\rm N}(\mathfrak p)^s)$ is defined to be $\chi({\rm Frob}_{\mathfrak p})$ when $\mathfrak p$ is a prime in $E$ that is unramified in the quadratic extension $E'$, while $\chi(\mathfrak p) = 0$ for $\mathfrak p$ in $E$ that are ramified in $E$.
Now apply the above decomposition of the zeta-function of the top field in a quadratic extension of a number fields twice to get the decomposition of the zeta-function of a biquadratic field: when $F = \mathbf Q(\sqrt{a},\sqrt{b})$ and $K = \mathbf Q(\sqrt{a})$, and $\chi$ is the nontrivial character of ${\rm Gal}(F/K)$, we have
$$
\zeta_F(s) = \zeta_K(s)L(s,\chi) = \zeta(s)L(s,\psi)L(s,\chi),
$$
where $\psi$ is the nontrivial character of ${\rm Gal}(K/\mathbf Q)$. Since $K/\mathbf Q$ is quadratic, you can view $\psi$ as a primitive Dirichlet character and that makes $L(s,\psi)$ above a Dirichlet $L$-function.
To show $L(s,\chi)$ is a product of two Dirichlet $L$-functions, rewrite the Euler product for $L(s,\chi)$ running over primes in $K$ as an Euler product over prime numbers $p$ by combining the Euler factors at primes in $K$ that lie over a common prime number $p$. Then you need to show the resulting quadratic Euler product over $\mathbf Q$ is the product of the Dirichlet $L$-functions of the two nontrivial characters of ${\rm Gal}(F/\mathbf Q)$ other than $\psi$.
Proving certain Euler factors from different computations agree could be particularly tricky for the factors at ramified primes, but keep in mind that Artin himself was unable at first to find the correct definition of Euler factors for Artin $L$-functions at ramified primes. First try to get the Euler factors on both sides of your desired formula to match up at prime numbers unramified in $F$ before looking at the case of primes that ramify.
Quite generally, as you know, the zeta-function of an abelian extension of $\mathbf Q$ is the product of Dirichlet $L$-functions for characters of the Galois group (viewed as a quotient group of some $(\mathbf Z/m\mathbf Z)^\times$ when the abelian extension is inside the $m$th cyclotomic field). This decomposition of a zeta-function as a product of Dirichlet $L$-functions can be proved without making any reference to Artin $L$-functions or representations beyond degree $1$. See Theorem 4.3 in Washington's book on cyclotomic fields. Some terminology in that proof (e.g., the "associated field") is introduced earlier in Chapter 3.