Is there an ordinal $\alpha$ such that $ZF$ believes that $V_{\alpha}$ is a model of $ZF$? (If it is problematic to state this since we have to check infinitely many axioms at once, formalize logic in $ZF$. ) If $\alpha > \omega$ is a limit ordinal, then $V_{\alpha}$ is a model of $ZF  R$, where $R$ stands for the axiom of replacement. The reflection principle tells us that $ZF$ knows that the set $S$ of ordinals $\alpha$ such that $R$ holds relative to $V_{\alpha}$ is unbounded. So is there some limit ordinal in $S$, thus answering the question? What is the smallest $\alpha$ such that $V_{\alpha}$ is a model of $ZF$, if it exists? I already know that if $\kappa$ is a strongly inaccessible cardinal, then $V_{\kappa}$ is a model of ZF, but the existence of such $\kappa$ is independent from $ZF$.
You cannot prove that there is such an ordinal, but (under a suitable large cardinal assumption) it is consistent that there is such an ordinal.
If you could prove that there was such an ordinal, then you will have proved Con(ZF) in ZF, contrary to the incompleteness theorem.
Another way to see it is: if there were such an ordinal, let $\alpha$ be the least ordinal with $V_\alpha\models$ZF. Thus, $V_\alpha$ is a model of ZF having no $\beta$ with $V_\beta\models$ZF, since the $V_\beta$ of $V_\alpha$ is the same as the $V_\beta$ of $V$.
However, if $\kappa$ is an inaccessible cardinal, then $V_\kappa\models$ZFC. In fact, there are many smaller $\alpha\lt\kappa$ with $V_\alpha\models$ZFC, and so the consistency strength of having an $\alpha$ with $V_\alpha\models$ZFC is strictly lower than an inaccessible, if it is consistent.
Your remark about using the Reflection Theorem to get $\alpha$ with $V_\alpha$ with Replacement is not quite right. The Replacement Axiom is a scheme of axioms, an infinite list of axioms, and the Reflection idea will only produce $\alpha$ with $V_\alpha$ satisfying any one (or finitely many) of them. But we cannot get a model of the whole scheme this way.
Lastly, it is interesting to note that every nonstandard model $M$ of ZF, having a nonstandard $\omega$, will have a $V_\alpha$ that is a model of ZF as viewed from outside $M$. The reason is that for any finite collection of the ZF axioms, we may apply the Reflection Theorem as you indicated to get a $V_\alpha^M$ satisfying them, but then since the $\omega$ of $M$ is nonstandard and $M$ cannot identify its standard cut, it follows by overspill that there must be some nonstandard finite set of ZF axioms in $M$ that $M$ thinks is satisfied in one of its $V_\alpha^M$. But since this includes all the standard axioms, we have thus obtained a $V_\alpha^M$ satisfying the true ZF as viewed externally.

2$\begingroup$ Martin: Here is a good exercise for you to think about: 1. In ZF we cannot prove that there is a model of ZF. 2. In " ZF + there is a model of ZF " we cannot prove there is a model of ZF with standard $\omega$ (an $\omega$model). 3. In " ZF + there is an $\omega$model of ZF" we cannot prove there is a wellfounded model. $\endgroup$ – Andrés E. Caicedo Sep 27 '10 at 15:05

$\begingroup$ What is meant by standard $\omega$ in ZF? $\endgroup$ – Martin Brandenburg Sep 27 '10 at 15:42

1$\begingroup$ Martin, by "a model of ZF with standard $\omega$" I mean: In the model $(M,E)$ there are a set $A$ and a set $B$ such that $(M,E)\models "A=\omega,B=\in\upharpoonright A^2"$. In $V$, we identify $A$ with the set $\hat A$ of those $a\in M$ such that $(M,E)\models a\in A$, and similarly with $B$. Then $(\hat A,\hat B)$ is what $(M,E)$ thinks is $\omega$. An $\omega$model or, equivalently, a model whose version of $\omega$ is standard, is one such that in $V$ we have: $(\hat A,\hat B)\cong(\omega,\in)$. One usually just codes this mouthful by saying that the model has no nonstandard integers. $\endgroup$ – Andrés E. Caicedo Sep 30 '10 at 0:15
Joel has thoroughly answered the question, but let me point out a related result that I think deserves to be better known. It's due to Montague and Vaught ["Natural models of set theories" Fund. Math. 47 (1959) 219242]. Suppose there is an inaccessible cardinal, and let $\delta$ be the first one. Define $\alpha$ to be the first ordinal such that $V_\alpha$ is a model of ZF. (As Joel pointed out, $\alpha<\delta$.) Let $\beta$ be the first ordinal such that $V_\beta$ is elementarily equivalent to $V_\delta$ (i.e., they satisfy the same firstorder sentences in the language of set theory). Let $\gamma$ be the first ordinal such that $V_\gamma$ is an elementary submodel of $V_\delta$ (i.e., any elements of $V_\gamma$ satisfy the same firstorder formulas in $V_\gamma$ as in $V_\delta$). Then the theorem of Montague and Vaught says that $\alpha<\beta<\gamma<\delta$ (with all inequalities strict).