$\newcommand{\al}{\alpha} \newcommand{\de}{\delta} \newcommand{\De}{\Delta} \newcommand{\ep}{\varepsilon} \newcommand{\ga}{\gamma} \newcommand{\Ga}{\Gamma} \newcommand{\la}{\lambda} \newcommand{\Si}{\Sigma} \newcommand{\thh}{\theta} \newcommand{\om}{\omega} \newcommand{\R}{\mathbb{R}} \newcommand{\Z}{\mathbb{Z}} \newcommand{\F}{\mathcal{F}} \newcommand{\E}{\operatorname{\mathsf E}} \renewcommand{\P}{\operatorname{\mathsf P}} \newcommand{\ii}[1]{\operatorname{\mathsf I}\{#1\}} \newcommand{\tf}{\widetilde{f}}$
This infimum is $\frac1\pi$. Indeed, integrating by parts, for any $f\in\F$ we have \begin{equation} \frac1{2\pi}=I(f):=\int_0^1 x f(x) dx =-J(f)/2,\quad J(f):=\int_0^1 x^2 f'(x) dx, \end{equation} whence \begin{equation} H(f):=\int_0^1 x^2 |f'(x)| dx\ge|J(f)|=\frac1\pi. \end{equation}\begin{equation} H(f):=\int_0^1 x^2 |f'(x)| dx\ge|J(f)|=\frac1\pi, \end{equation} so that the infimum in question is $\ge\frac1\pi$.
On the other hand, for any $\ep\in(0,1)$ let \begin{equation} f_\ep:=c_\ep g_\ep,\quad g_\ep(x):=\frac x\ep\,1_{(0,\ep]}(x)+\frac{1-x}{1-\ep}\,1_{(\ep,1)}(x), \end{equation} where $c_\ep$ is such that $I(f_\ep)=\frac1{2\pi}$, so that $c_\ep\to3/\pi$; the convergence everywhere here is for $\ep\downarrow0$. Then $H(f_\ep)\to\frac1\pi$. It remains to approximate in $L^1[0,1]$, however closely, both $f_\ep$ and $f'_\ep$ by $f$ and $f'$ for some $f\in\F$; this can be done by using, say, the convolution of $f_\ep$ (or, rather, of a close version of $f_\ep$ with support on $(0,1)$) with a smooth kernel and the multiplication by a constant factor close to $1$. So, the infimum in question is indeed $\frac1\pi$.