Let $k_n$ be the least integer $k$ such that, for any digraph $D$ of order $n$ and any vertices $x,y\in D$, if there are $x$-$y$ walks of length $1,\dots,k$, then there are $x$-$y$ walks of all positive integer lengths.
Theorem. $k_1=1$ and, for $n\ge2$,
$$k_n=n-1+\max\{g(m)-m:0\le m\le n-2\}$$
where g(m) is Landau's function, the maximum order of an element in the symmetric group $\mathbf S_m$.
(So $k_n=n$ for $n\le8$, i.e., YCor's example with $n=9$ vertices is as small as possible.)
Trivially $k_1=1$; we consider $n\ge2$. I have to show that
$$k_n=f(n):=n-1+\max\{g(m)-m:0\le m\le n-2\}.$$
Note that $f(n)\ge n$ since $g(0)=1$.
To show that $k_n\le f(n)$ it will suffice to prove the following:
Lemma 1. Suppose $x,y$ are vertices in a digraph of order $n\ge2$, and suppose $k\ge f(n)$. If there are $x$-$y$ walks of length $1,\dots,k$, then there is an $x$-$y$ walk of length $k+1$.
Proof. Assume for a contradiction that there are $x$-$y$ walks of length $1,\dots,k$ but none of length $k+1$.
Note that no cycle contains $x$ or $y$. For if $C$ were a cycle of length $m$ containing $x$ or $y$, then by adding $C$ to an $x$-$y$ walk of length $k+1-m$ we would get an $x$-$y$ walk of length $k+1$.
Let $C_1,\dots,C_t$ be a maximal family of (vertex-)disjoint cycles with respective lengths $m_1,\dots,m_t$. Let $m=m_1+\cdots+m_t\le n-2$ and let $L=\operatorname{LCM}(m_1,\dots,m_t)\le g(m)$. Now
$$k\ge f(n)\ge n-1+g(m)-m\ge n-1+L-m,$$
whence
$$k\ge k+1-L\ge n-m\ge2.$$
Let $W$ be an $x$-$y$ walk of length $k+1-L$. Then $W$ is disjoint from the cycles $C_1,\dots,C_t$, as otherwise there would be an $x$-$y$ walk of length $k+1$. So $W$ has at most $n-m$ vertices, and length $k+1-L\ge n-m$, so $W$ contains a cycle, which is disjoint from $C_,\dots,C_t$, contradicting the assumed maximality of that family of disjoint cycles.
We have shown that $k_n\le f(n)$. We will show $k_n\ge f(n)$ by generalizing YCor's construction.
Lemma 2. $k_{n+1}\ge k_n+1$.
Proof. Take a digraph of order $n$ with vertices $x,y$ such that there are $x$-$y$ walks of length $1,\dots,k_n-1$ but none of length $k_n$. Add a vertex $y'$ and an arc from $y$ to $y'$. In the resulting digraph of order $n+1$ there are $x$-$y'$ walks of length $1,\dots,k_n$ but none of length $k_n+1$.
Lemma 3. $k_n\ge g(n-2)+1$ for $n\ge2$.
Proof. Write $g(n-2)=\operatorname{LCM}(m_1,\dots,m_t)$ where $m_1+\cdots+m_t=n-2$. Consider a digraph consisting of disjoint cycles $C_1,\dots,C_t$ of respective lengths $m_1,\dots,m_t$ and two additional vertices $x$ and $y$. Draw an arc from $x$ to $y$; draw arcs from $x$ to one vertex in each cycle; and draw arcs to $y$ from all but one vertex in each cycle, chosen so that there is an $x$-$y$ walk through $C_i$ of length $k$ iff $k\not\equiv1\pmod{m_i}$. Thus there is an $x$-$y$ walk of length $k$ iff either $k=1$ or else $k\not\equiv1\pmod{g(n-2)}$.
We finish the proof of $k_n\ge f(n)$ by proving the following:
Lemma 4. If $0\le m\le n-2$ then $k_n\ge n-1+g(m)-m$.
Proof. By Lemmas 2 and 3, since $n\ge m+2$,
$$k_n\ge n-(m+2)+k_{m+2}\ge n-m-2+g(m)+1$$
$$=n-1+g(m)-m.$$