The answer depends upon, of course, if the players make their moves intelligently or stupidly. In either case, build an alpha-beta tree of move sequences and search either depth-first or breadth-first, stopping and backtracking as winning configurations are found. However, there are some rigorous points you have not defined about this question, including whether perhaps it came to you as a homework problem. Hoping that this is not the case, here's a take on it:
Yes, if $n=2$, then the first player to move obviously wins by picking any one of the four open spots, and the game is over. If $n=3$ (which you don't define for odd $n$, but lets say for odd $n$, $(n+1)/2$ spaces wins, then the first player to move wins automatically also, because no matter which space they take in the $3 \times 3$ grid, the next player can only block one space, and in the 2nd round, first player easily wins.
For $n=4$, the first player wins, with the same strategy as shown for $n=3$. $A$ picks any square in the $4 \times 4$ grid, $B$ picks any other square but can only block $A$ in one of the two dimensions in which this grid lies, therefore on the first step of the second round, $A$ wins by picking a square adjacent to his first square.
Are you sure that you've thought this out for the smaller grid games?
So for the $4 \times 4$ configuration, assuming that $A$ plays intelligently, there are
$4 \times 4 = 16$ first moves for $A$
$16 - 1 = 15$ first moves for $B$, if $B$ plays stupidly, or either $2, 3,$ or $4$ moves if $B$ plays intelligently
$4 \times 1 + 8 \times 2 + 4 \times 3 = 4+16+12=32$ second moves for $A$ which win the game and end the game ($A$ could have picked one of the $4$ corners, $B$ blocks one way leaving $1$ way for $A$ to play, if $A$ picks one of the non-corner edge pieces (8 ways to do that), then $B$ blocks one way leaving two ways for $A$'s second winning move, and if $A$'s first move is a center piece, $B$ can block one of the $4$ adjacent squares, leaving $3$ adjacent squares for $A$ to pick and win.
So if $A$ and $B$ both play intelligently, the sum of these are the possible games for a $4 \times 4$ grid:
$4 \cdot 2 \cdot 1 + 8 \cdot 3 \cdot 2 + 4 \cdot 4 \cdot 3 = 8 + 48 + 48 = 104$
If $A$ and $B$ do not play intelligently, then the number of possible games is much larger, and the number of "winning configurations" or "end positions" is much larger.