This question has been out for a while, and I think it deserves a thorough answer, even tho I came quite late to this party.
As both the classical Legendre-Jacobi theory and the Carlson theory have been mentioned, I'll treat the OP's integral in both viewpoints.
Legendre-Jacobi
The OP came pretty close to using the correct substitution. One thing that could have been done instead is to recall the Pythagorean identity $1+\tan^2 u=1$, so that the substitution that should have been used is $x=a \tan^2 u$, where $a=2$ or $a=3$. Taking the smaller value of $a$, and after some amount of algebra, we obtain
$$\begin{align*}\int_0^\infty\frac{\mathrm dx}{\sqrt{x(x+2)(x+3)}}&=\int_0^{\pi/2}\frac{2}{\sqrt{3-\sin^2u}}\mathrm du\\ &=\frac2{\sqrt{3}}\int_0^{\pi/2}\frac{\mathrm du}{\sqrt{1-\frac13\sin^2u}}\\&=\frac2{\sqrt{3}}K\left(\frac13\right)\end{align*}$$
where I use the parameter convention for elliptic integrals. (Relatedly, see here for an extended discussion on the notational confusion surrounding elliptic integrals.)
Had we chosen the substitution with $a=3$ instead, we would have instead obtained the result $\sqrt{2}K\left(-\frac12\right)$, which is equivalent through the imaginary modulus transformation
$$K(-m)=\frac1{\sqrt{1+m}}K\left(\frac{m}{m+1}\right)$$
Carlson
In Carlson's theory, there is the general hypergeometric function
$$R_{-a}(b_1,\dots,b_k;z_1,\dots,z_k)=\frac1{\mathbf B\left(a,-a+\sum_j b_j\right)}\int_0^\infty u^{-a-1+\sum_j b_j}\prod_j \left(u+z_j\right)^{-b_j}\mathrm du$$
where $\mathbf B(x,y)=\frac{\Gamma(x)\Gamma(y)}{\Gamma(x+y)}$ is the usual Euler beta function.
This multivariable hypergeometric function is a homogenized/symmetrized version of the classical Lauricella $F_D$ function (see e.g. this paper where Carlson introduced his function, tho that reference uses an opposite sign convention for $a$).
With this consideration, the OP's original integral can be expressed either as a three-variable Carlson integral, or a two-variable Carlson integral:
$$\begin{align*}\int_0^\infty\frac{\mathrm dx}{\sqrt{x(x+2)(x+3)}}&=2\,R_{-\frac12}\left(\frac12,\frac12,\frac12;0,2,3\right)\\&=\pi\,R_{-\frac12}\left(\frac12,\frac12;2,3\right)\end{align*}$$
The three-variable (incomplete) case occurs often enough that it is given the notation
$$\begin{align*}R_F(x,y,z)&=\frac12\int_0^\infty\frac{\mathrm du}{\sqrt{(u+x)(u+y)(u+z)}}\\&=R_{-\frac12}\left(\frac12,\frac12,\frac12;x,y,z\right)\end{align*}$$
and the two-variable form is the so-called "complete case",
$$\begin{align*}R_K(x,y)&=R_{-\frac12}\left(\frac12,\frac12;x,y\right)\\&=\frac2{\pi}R_F(0,x,y)\end{align*}$$
In fact, the two-variable form has the integral representation
$$R_K(x,y)=\frac2{\pi}\int_0^{\pi/2}\frac{\mathrm du}{\sqrt{x\cos^2 u+y\sin^2 u}}$$
which one recognizes to be related to the integral representation of Gauss's arithmetic-geometric mean (AGM):
$$R_K(x,y)=\frac1{\operatorname{agm}(\sqrt{x},\sqrt{y})}$$
Thus, the OP's integral is $\pi/\operatorname{agm}(\sqrt{2},\sqrt{3})$.
Additionally, the two-variable Carlson function is also related to the Gauss hypergeometric function, being a homogeneous version of it:
$$R_{-a}(b_1,b_2;x,y)=y^{-a}{}_2 F_1\left({{a,b_1}\atop{b_1+b_2}}\middle|1-\frac{x}{y}\right)$$
so one has
$$\begin{align*}\pi\,R_{-\frac12}\left(\frac12,\frac12;2,3\right)&=\frac{\pi}{\sqrt{3}}{}_2 F_1\left({{\frac12,\frac12}\atop{1}}\middle|\frac13\right)\\&=\frac2{\sqrt{3}}K\left(\frac13\right)\end{align*}$$
where we have used the hypergeometric representation of the complete elliptic integral of the first kind.