I tried calculus of variations. It's been a while since I've done calculus of variations, so I could be messing it up completely. Also, this isn't completely rigorous. There are ways of making calculus of variations rigorous, but I don't know them, and they're a lot harder than just doing calculations.
By my comment above, the $Q(\frac{1}{2})=0$ constraint is worthless, so we will ignore it.
Assume $Q$ is the minimum function (this is the first, and most important, non-rigorous step, since we are assuming a minimum exists). Now, let's plug in $Q+\epsilon$, where $Q$ and $\epsilon$ are both functions of $x$. We have
$$M=\frac{1}{12} + \int_0^\frac{1}{2} \left( \tfrac{1}{2}-x \right)^2 (Q+\epsilon)^2 \ dx - 4 \left[ \int_0^\frac{1}{2} \left( \tfrac{1}{2}-x \right) (Q+\epsilon) \ dx\right]^2 .$$
Extracting the first-order terms in $\epsilon(x)$, we get
$$\Delta M = 2\int_0^\frac{1}{2} \left( \tfrac{1}{2}-x \right)^2 Q \epsilon \ dx -8 \int_0^\frac{1}{2} \left( \tfrac{1}{2}-x \right) Q \ dx \ \int_0^\frac{1}{2} \left( \tfrac{1}{2}-x \right) \epsilon \ dx .$$
If we change $Q$ by adding an infinitessimal $\epsilon$, and keeping the normalization condition on $Q$, then $\Delta M$ has to be 0, or otherwise $Q$ wouldn't be a minimum. We still have to take care of the normalization condition $\int_0^\frac{1}{2} Q(x)^2 \ dx = 1$. We do this using Lagrange multipliers, and so we get the expression
$$2\int_0^\frac{1}{2} \left( \tfrac{1}{2}-x \right)^2 Q \epsilon \ dx -8 \int_0^\frac{1}{2} \left( \tfrac{1}{2}-x \right) Q \ dx \ \int_0^\frac{1}{2} \left( \tfrac{1}{2}-x \right) \epsilon \ dx + \lambda \int_0^\frac{1}{2} Q \epsilon\ dx .$$
This has to be zero for all functions $\epsilon(x)$. To simplify things further, let
$$C = \int_0^\frac{1}{2} \left( \tfrac{1}{2}-x \right) Q \ dx,$$
since it's a constant independent of $\epsilon$. Now, we have
$$2 \int_0^\frac{1}{2} \left( \tfrac{1}{2}-x \right)^2 Q \epsilon \ dx -8 C \ \int_0^\frac{1}{2} \left( \tfrac{1}{2}-x \right) \epsilon \ dx + \lambda \int_0^\frac{1}{2} Q \epsilon\ dx $$
or, putting everything in the same integral sign,
$$ \int_0^\frac{1}{2} 2 \left( \tfrac{1}{2}-x \right)^2 Q \epsilon -8 C \left( \tfrac{1}{2}-x \right) \epsilon + \lambda Q \epsilon\ dx $$
and for this to be $0$ for all functions $\epsilon$, we must have
$$Q = \frac{ (\frac{1}{2}-x)}{\alpha(\frac{1}{2}-x)^2+\beta}$$
for some $\alpha$, $\beta$.
Note that, if $\beta \neq 0$, we get $Q(\frac{1}{2}) = 0$, as desired.
A Maple or Mathematica program should at least let you calculate $\alpha$ and $\beta$ numerically. When I do this, I seem to be getting values of $M$ which are slightly less than 0, but I guess I could have mistyped my Maple formulas.