The box dimension is not countably stable, so even one exceptional point could drive it up. For example, the function $f(x)=\sqrt{x}sin(1/x^2)$ on $[0,1]$, with $f(0)=0$, is locally Lipschitz on $(0,1]$, yet the graph of $f$ has box dimension at least $1.25$. Indeed, for $j<k$, to cover the graph above $[1/(2\pi(k+j)), 1/(2\pi(k+j+1))]$, at least $ck^{3/2}$ squares of side length $1/(4k^2)$ are needed. Summing over $j<k$ we get $c_1 k^{5/2}$ squares of side length $1/(4k^2)$.
You can get a result of the type you want by working with the Hausdorff or packing dimension of the graph, as these notions are stable under countable union.
In order to prove the packing dimension of the graph of $f$ is at most $2-\alpha$, it suffices to assume that the exceptional set (where the local$\alpha$-H"older condition fails to hold) has box dimension at most $1-\alpha$. [This condition is sharp, unless you make further assumptions on the behavior of $f$ on the exceptional set.]
To see this, partition the graph of $f$ to the part over the exceptional set, and for each integer $m$, the part where the $\alpha$-H"older condition holds with constant $m$. Then use the connection between packing dimension and Box dimension to conclude.