Open interval $(a,b)$ easy ... make $f'$ unbounded, say $f(x) = \sqrt{x}$ on $(0,1)$.

Requiring differentiability even at the endpoint, the counterexample must be more elaborate.  But still an unbounded function is not Riemann integrable, so take some $x^a \sin^b x$.

Even allowing improper Riemann integrals or Lebesgue integral is not enough to avoid the hypothesis that $f'$ is integrable.  The Henstock-Kurzweil integral is needed to recover $f$ from $f'$ which exists everywhere on $[a,b]$ in general.