Gromov--Witten invariants are designed to count the "number" of curves in a space in a deformation invariant way. Since the number of curves can change under deformations, the Gromov--Witten invariants won't have a direct interpretation in terms of actual numbers of curves, even taking automorphisms into account.
Here is an example of how a negative number might come up, though strictly speaking it isn't a Gromov--Witten invariant. Let M be the moduli space of maps from P^1 to a the total space of O(-4) on P^1. Call this space X. Note that I said maps from P^1, not a genus zero curve, so the source curve is rigid. That's why this isn't Gromov--Witten theory. Any such map factors through the zero section (since O(-4) has no nonzero sections), so this space is the same as the space of maps from P^1 to itself. I just want to look at degree one maps, so the moduli space is 3 dimensional.
We could also compute the dimension using deformation theory: the deformations of a map f are classified by $H^0(f^\ast T)$ where T is the tangent bundle of the target. The target in this case is O(-4), not just P^1, and the tangent bundle restricts to O(2) + O(-4) on the zero section. Thus $H^0(f^\ast T)$ is indeed 3-dimensional, as we expected. However, the Euler characteristic of $f^\ast T$ is not 3 but 0, which means that the "expected dimension" is zero.
The meaning of expected dimension is rather vague. Roughly speaking, it is the dimension of the moduli space for a "generic" choice of deformation. The trouble is that such a deformation might not actually exist. Nevertheless, we can still pretend that a generic deformation does exist and, if the expected dimension is zero, compute the number of curves that it "should" have.
What makes this possible is the obstruction bundle E on M. Any deformation of X gives rise to a section of E and the vanishing locus of this section is the collection of curves that can be deformed to first order along with X. Even though a generic deformation might not exist, the obstruction bundle does still exist, and we can make sense of the vanishing locus of a generic section by taking the top Chern class.
In our situation, the (fiber of the) obstruction bundle is $H^1(f^\ast T)$. Since O(2) does not contribute to H^1, the obstruction bundle is $R^1 p_\ast f^\ast O_{P^1}(-4) = R^1 p_\ast O_{P^3 \times P^1}(-4, -4)$ where $p : P^3 \times P^1 \rightarrow P^3$ is the projection. By the projection formula, this is $O(-4)^{\oplus 3}$ and the top Chern class is -64. This is the "Gromov--Witten invariant" of maps from P^1 to $O_{P^1}(-4)$.
Unfortunately, I don't have anything to say about what this -64 means...