(This is not an answer, rather an extended comment.)

If $X = a \frac{n}{n-1}$ with probability $\frac{n-1}{n}$ and $X = 0$ otherwise, then $x_1 = \mathbb{E}X = a$, $x_2 = \mathbb{E}X^2 = a^2 \frac{n}{n-1}$ and $x_3 = \mathbb{E}X^3 = a^3 (\frac{n}{n-1})^2$, so that $$\frac{x_3 + x_1^3 - 2 x_1 x_2}{(x_2 - x_1^2)^2} = \frac{1}{a} .$$
Of course, one can smooth out $X$ a little bit to get an absolutely continuous distribution with the above ratio arbitrarily close to $\frac{1}{a}$.

My guess would be that $\dfrac{1}{a}$ is the lower bound for $\dfrac{x_3 + x_1^3 - 2 x_1 x_2}{(x_2 - x_1^2)^2}$ if $x_1$ is required to be equal to $a$.

Let $X$ have density function $f(x)$, and let $Y = X/a - 1$, so that $Y \geqslant -1$ and $\mathbb{E} Y = 0$ (recall that we assume that $x_1 = a$). Observe that
$$x_3 + x_1^3 - 2 x_1 x_2 = \mathbb{E}(X^3 + a^3 - 2 a X^2) = a^3 \mathbb{E}(Y^2 + Y^3)$$
and
$$ x_2 - x_1^2 = \mathbb{E}(X^2 - a^2) = a^2 \mathbb{E} Y^2 . $$
Thus,
$$ \frac{x_3 + x_1^3 - 2 x_1 x_2}{(x_2 - x_1^2)^2} - \frac{1}{a} = \frac{1}{a} \, \frac{\mathbb{E}Y^2 + \mathbb{E}Y^3}{(\mathbb{E}Y^2)^2} - \frac{1}{a} = \frac{1}{a} \, \frac{\mathbb{E}(Y^2 (1 + Y)) - (\mathbb{E}Y^2)^2}{(\mathbb{E}Y^2)^2} . $$
My guess is therefore equivalent to
$$ \mathbb{E}(Y^2 (1 + Y)) \geqslant (\mathbb{E}Y^2)^2 $$
whenever $\mathbb{E} Y = 0$ and $Y \geqslant -1$.

I do not an immediate proof of the above inequality, nor do I see a counter-example. I though I would share it anyway, perhaps someone else can help. *Edit: the proof is completed in Iosif Pinesis's answer.*