Clearly it suffices to prove that for any $a<b$ we have $\DeclareMathOperator{\Lip}{Lip}$
$$
\int_a^b \big|\; f-L(f)\;\big| dx\leq \frac{b-a}{4}\Lip(f),
$$
where $L(f)$ is the linear function that connects the endpoints of the graph of $f$
$$
x\mapsto f(a)+\frac{f(b)-f(a)}{b-a}x.
$$
After rescaling and translations we can assume $a=0$, $b=1$, $f(a)=0$ so it suffices to prove
$$
\int_0^1\big|\; f(x) -f(1) x\;\big| dx\leq \frac{1}{4}\Lip(f),
$$
$\forall f\in \Lip([0,1])$, $f(0)=0$.
We have
$$
\int_0^1\big|\; f(x) -f(1) x\;\big| dx=\int_0^1\Bigg|\; \int_0^x\big(\;f'(y)-f(1)\;\big)dy\;\Bigg| dx
$$
$$
= \int_0^1 \Bigg|\; \int_0^xf'(y)-f(1)\;dy\;\Bigg|\; dx\leq \int_0^1 \int_0^x \big|\;f'(y)-f(1)\;\big|\;dy\; dx
$$
$$
= \int_0^1 \int_y^1\big|\;f'(y)-f(1)\;\big|dx\; dy= \int_0^1 (1-y)\;\big|\;f'(y)-f(1)\;\big|\;dy
$$
Now observe that $\DeclareMathOperator{\Mean}{Mean}$
$$
f(1)=\Mean(f').
$$
We need to find the optimal constant $C$ in the inequality
$$
\Big\Vert g-\Mean(g)\Big\Vert_{L^\infty}\leq C\Vert g\Vert_{L^\infty},\;\;g\in L^\infty(0,1).
\tag{1}
$$
It is not very hard to see that $C\in [1,2]$. We deduce
$$
\int_0^1\big|\; f(x) -f(1) x\;\big| dx\leq \frac{C}{2}\Vert f'\Vert_\infty.
$$
Another approach.
$$
\int_0^1 (1-y)\;\big|\;f'(y)-f(1)\;\big|\;dy\leq \left(\int_0^1 (1-y)^2 dy\right)^{1/2}\left(\int_0^1 (f'(y)-f(1))^2dy\right)^{1/2}.
$$
On the other hand, for any $g\in L^2(0,1)$ we have
$$
\int_0^1\Big(\; g(y)-\Mean(g)\;\big)^2 dy\leq \int_0^1 g(y)^2 dy \leq \Vert g\Vert_{L^\infty}^2.
$$
Hence
$$
\int_0^1 (1-y)\;\big|\;f'(y)-f(1)\;\big|\;dy\leq \left(\int_0^1 (1-y)^2 dy\right)^{1/2}\Vert f'\Vert_{L^\infty}=\frac{1}{\sqrt{3}}\Lip(f).
$$
I don't know yet how to improve the constant $\frac{1}{\sqrt{3}}$ to $\frac{1}{4}$.