Here's an example of size 9 inspired domotorp's anwer, where I only consider the primes $2,5$, and where I replace the additional long arrows from the basis of the cycles with arrows from the vertices themselves, which avoids additional vertices, and where I change the congruences to precisely forbid $n+1=10$ instead of $11$.

In words: I have 9 vertices: the initial and terminal vertices $x,y$, a $2$-cycle $v_2=\{b_0,b_1\}$, a $5$-cycle $v_5=\{c_0,\dots,c_4\}$. Join: $x\to y$, $x\to b_O$, $x\to c_0$, $b_1\to y$, $c_i\to y$ for $i=0,1,2,4$ (i.e. not for $i=3$), and the cycles $b_0\to b_1\to b_0$, $c_0\to c_1\to c_2\to c_3\to c_4\to c_0$.

Then for $m\ge 0$ there is a path from $x$ to $y$ of length $m$ iff $m$ is nonzero modulo $10$.

Here's a picture of the digraph:

Here's an example of size 9 inspired by domotorp's anwer, where I only consider the primes $2,5$, and where I replace the additional long arrows from the basis of the cycles with arrows from the vertices themselves, which avoids additional vertices, and where I change the congruences to precisely forbid $n+1=10$ instead of $11$.

In words: I have 9 vertices: the initial and terminal vertices $x,y$, a $2$-cycle $v_2=\{b_0,b_1\}$, a $5$-cycle $v_5=\{c_0,\dots,c_4\}$. Join: $x\to y$, $x\to b_O$, $x\to c_0$, $b_1\to y$, $c_i\to y$ for $i=0,1,2,4$ (i.e. not for $i=3$), and the cycles $b_0\to b_1\to b_0$, $c_0\to c_1\to c_2\to c_3\to c_4\to c_0$.

Then for $m\ge 0$ there is a path from $x$ to $y$ of length $m$ iff $m$ is nonzero modulo $10$.

Here's a picture of the digraph: