I will just consider the simplest example, maybe someone will give the answer in complete generality. So let $M=\mathbb CP^n$. Then $4n-2c_1(M)(A)=2n-2$. This means that we are in good shape. Basically we can take for $X$ and $Y$ any complex submanifolds of $M$ satisfying your condition. Indeed in this case for a generic point $p$ in $\mathbb CP^n$ there will be $deg X\cdot deg Y$ lines that contain $p$ and intersect both $X$ and $Y$.

I will just consider the simplest example, maybe someone will give the answer in complete generality. So let $M=\mathbb CP^n$. Then $4n-2c_1(M)(A)=2n-2$. This means that we are in good shape. Basically we can take for $X$ and $Y$ any complex submanifolds of $M$ satisfying your condition. Indeed in this case for a generic point $p$ in $\mathbb CP^n$ there will be $deg X\cdot deg Y$ lines that contain $p$ and intersect both $X$ and $Y$. I assumed $X$ (or $Y$) is not zero dimensional, in which case there is only one line and also that $X$ and $Y$ are in general position, but this does not matter for GW, of course.