Yes, the Taylor series works. Actually $C^2$ suffices for the remainder term, although my sophomore calculus book gives the proof using $C^3.$ I get $$ 4 f(x_0, y_0) = 4 f(x_0, y_0) + \left( 2 f_{xx}(x_0, y_0) + 2 f_{yy}(x_0, y_0) \right) \delta^2 \; + \; o( \delta^2 ) $$ and $$ \left( 2 f_{xx}(x_0, y_0) + 2 f_{yy}(x_0, y_0) \right) \delta^2 \; = \; o( \delta^2 ) $$ and $$ 2 \left( f_{xx}(x_0, y_0) + f_{yy}(x_0, y_0) \right) \; = \; 0 $$
Well, I will post this, hope it was not homework.