Your belief is not correct, as Robert Israel pointed in his comment. Same for differential equations: $yy'-(3/2)z^2=0$ has a solution $y(z)=z^{3/2}$ which is not analytic at $0$. The correct theorem is: If $P(z_0,y_0,y_1,\ldots,y_{n})=0$, where $P$ is a polynomial, (or an analytic function in a neighborhood of $(z_0,\ldots,y_n)$), and $\partial P/\partial y_{n}\neq 0$ at the same point, then there exists a unique analytic function $y(z)$ in a neighborhood of $z_0$, which satisfies $P(z,y,y',\ldots,y^{(n)})=0$ in this neighborhood, and $y(z_0)=y_0,\; y'(z_0)=y_0,\ldots,y^{(n)}(z_0)=y_{n}$. One does not need to assume a priori that $y(z)$ is analytic. If it is $n$ times differentiable in a neighborhood of $z_0$, it is automatically analytic. For the proof, see, for example H. Cartan, Elementary theory of analytic functions of one and several complex variables, or Coddington and Levinson, Ordinary differential equations.
EDIT. The second question (asked in the comment) seems to be: if $f(z)$ is a formal power series satisfying a polynomial equation, does it follow that $f$ is convergent (so $f$ is analytic). The answer is yes in the case of implicit function theorem ($P(z,f(z))=0$), and no in the case of differential equations: the divergent series $$f(z)=\sum_{n=1}^\infty (n-1)!z^n$$ satisfies the Euler equation $z^2f'(z)-f(z)-z=0$, and $f(0)=0$.
