It is a standard fact that $L(s,\chi)$, initially defined as a locally uniformly convergent Dirichlet series in $\Re s>1$, extends to a holomorphic function on $\mathbb{C}$. See for example Chapter 9 in Davenport: Multiplicative Number Theory, especially the first half of Page 69.
Now basic complex analysis tells us that the Taylor series of $L(s,\chi)$ around $s=1$ converges absolutely on $\mathbb{C}$, of which $$ L(s,\chi)=L(1,\chi)+O(|s-1|) $$ is a consequence. In fact for the last equation we only need to know that $L'(s,\chi)$ is continuous on $\mathbb{C}$, which follows directly from Cauchy's integral formula.
A quick proof of the holomorphicity of $L(s,\chi)$ in $\Re s>0$ follows from the fact that the partial sums $\sum_{n\leq x}\chi(n)$ are bounded. See Proposition 9 in Section VI.2 of Serre: A course in arithmetic, or Theorem 1.3 in Montgomery-Vaughan: Multiplicative number theory I.
