One can take the continuum limit of your proof as $X \to \infty$, again using the prime number theorem, to obtain a proof that does not involve primes at all:
$$ \int f(t) \log \frac{1}{|t|}\ dt = \gamma - \sum_{\sigma = \pm 1} \int_0^\infty f(\sigma t) (\log t + \gamma)\ dt $$ $$ = \gamma - \lim_{\varepsilon \to 0} \sum_{\sigma = \pm 1} \int_0^\infty f(\sigma t) (\log \frac{t}{\varepsilon} + \gamma)\ dt + \log \frac{1}{\varepsilon}$$ $$ = \gamma - \lim_{\varepsilon \to 0} \sum_{\sigma = \pm 1} \int_0^\infty f(\sigma t) (\sum_{0 < s < t/\varepsilon} \frac{1}{s})\ dt + \log \frac{1}{\varepsilon}$$ $$ = \gamma - \lim_{\varepsilon \to 0} \sum_{\sigma = \pm 1} \sum_{s>0} \frac{1}{s} \int_{\varepsilon s}^\infty f(\sigma t)\ dt + \log \frac{1}{\varepsilon}$$ $$ = \gamma - \lim_{\varepsilon \to 0} \sum_{\sigma = \pm 1} \sum_{s>0} \int_{\varepsilon}^\infty f(\sigma s t)\ dt + \log \frac{1}{\varepsilon}$$ $$ = \gamma - \lim_{\varepsilon \to 0} \int_{\varepsilon}^\infty \sum_{s \in \mathbb{Z} \backslash \{0\}} f(s t)\ dt + \log \frac{1}{\varepsilon}$$ $$ = \gamma - A - \lim_{\varepsilon \to 0} \int_{\varepsilon}^1 \sum_{s \in \mathbb{Z} \backslash \{0\}} f(s t)\ dt + \log \frac{1}{\varepsilon}$$ $$ = \gamma - A - \lim_{\varepsilon \to 0} \int_{\varepsilon}^1 (\sum_{s \in \mathbb{Z} \backslash \{0\}} f(s t) - \frac{1}{t})\ dt $$ $$ = \gamma - A - \lim_{\varepsilon \to 0} \int_{\varepsilon}^1 (\frac{1}{t} \sum_{s \in \mathbb{Z} \backslash \{0\}} \hat f(s/t) - 1)\ dt $$ $$ = \gamma + 1 - A - B$$ where $$ A := \int_1^\infty \sum_{s \in \mathbb{Z} \backslash \{0\}} f(st)\ dt$$ $$ = \int_{|t| \geq 1} f(t) (\sum_{1 \leq s \leq |t|} \frac{1}{s})\ dt$$ and $$ B := \int_0^1 \sum_{s \in \mathbb{Z} \backslash \{0\}} \hat f(s/t)\ \frac{dt}{t}$$ $$ = \int_1^\infty \sum_{s \in \mathbb{Z} \backslash \{0\}} \hat f(st)\ \frac{dt}{t}$$ $$ = \int_{|t| \geq 1} \hat f(t) \frac{\lfloor |t| \rfloor}{t}\ dt.$$ [In the language of distributions, what this identity is saying I think is that the distributional Fourier transform of $\lfloor |t| \rfloor/t - \gamma \delta$ is $1 - \log \frac{1}{|t|} 1_{|t| \leq 1} - \sum_{1 \leq s \leq |t|} \frac{1}{s}$, where $\delta$ is the Dirac delta.]
Since $A,B$ are clearly non-negative, this gives your inequality. This also shows that one is within $o(1)$ of equality if and only if one simultaneously has $$ \int_{|t| \geq 1} f(t) (1 + \log |t|)\ dt = o(1)$$ and $$ \int_{|t| \geq 1} \hat f(t)\ dt = o(1).$$ By the Hahn-Banach theorem, these estimates are incompatible with the hypotheses $f(0)=\hat f(0)=1$, $f(t), \hat f(t) \geq 0$ for $|t| \geq 1$ if and only if there exist non-negative measurable functions $a(t), b(t)$ supported on $|t| \geq 1$ with $\sup_t \frac{a(t)}{1+\log |t|}, \sup_t b(t) < \infty$ and numbers $\alpha,\beta$ not summing to zero, such that $$ \alpha f(0) + \beta \hat f(0) = \int_{\mathbb R} f(t) a(t)\ dt + \int_{\mathbb R} \hat f(t) b(t)\ dt $$ for all Schwartz $f$, or equivalently that $$ \alpha \delta + \beta = a + \check b$$ in the sense of tempered distributions, where $\delta$ is the Dirac delta. But the right-hand side is continuous at the origin, so $\alpha$ must vanish; the Fourier transform of the right-hand side has a continuous antiderivative at the origin, so $\beta$ must vanish, contradiction. This shows that one can make $A$ and $B$ simultaneously $o(1)$, so $1+\gamma$ is in fact optimal. (But the invocation of the Hahn-Banach theorem makes it difficult to explicitly construct $f$ that come close to equality!)