This is not true: it does not need to be connected even if $X$ is smooth connected over an algebraically closed field $K$ of characteristic zero. Indeed, if there is an isogeny $\pi \colon A \to B$ and a (necessarily smooth and connected) subscheme $Y \subseteq B$ such that $X = \pi^*Y$ (the scheme-theoretic inverse image), then $a \in A(K)$ fixes $X$ if and only if $\pi(a)$ fixes $Y$, so we get a short exact sequence
$$0 \to \ker \pi \to \operatorname{Stab}_A(X) \to \operatorname{Stab}_B(Y) \to 0.$$
(Exactness on the right follows from surjectivity of $\pi$.)

But example of such $X$ are aplenty; for instance we can start with a nontrivial isogeny $\pi \colon A \to B$ and take $Y \subseteq B$ a smooth complete intersection of dimension $\geq 1$ of (very) ample divisors. Then $X$ is a complete intersection of ample divisors [Hart, Exc. III.5.7], so $X$ is connected [Hart, Cor. III.5.7]. Since $Y$ is smooth and $X \to Y$ is finite étale, $X$ is smooth as well (see for instance Tag [01VA](https://stacks.math.columbia.edu/tag/01VA)).

---

**Remark.** However, it is true that the stabiliser is a smooth closed subgroup of $A$ that is defined over $K$, so its identity component is an abelian variety. Here's how to set this up scheme-theoretically:

We should first say what is being stabilised. I think the natural candidate is the point $X$ in the Hilbert scheme $\mathbf{Hilb}_A$ of closed subschemes of $A$ (see for instance [FGAE, Ch. 5]). The multiplication action $A \times A \to A$ of $A$ on itself naturally defines an action of $A$ on $\mathbf{Hilb}_A$ using the functor of points, and one considers the stabiliser of $X \in \mathbf{Hilb}_A(K)$, i.e. the pullback
$$\begin{array}{ccc}\operatorname{Stab}_A(X) & \to & \mathbf{Hilb}_A \\ \downarrow & & \downarrow \\ A & \to & \mathbf{Hilb}_A \times \mathbf{Hilb}_A,\!\end{array}$$
where the bottom arrow is given on functor of points by $a \mapsto (X,aX)$ and the right vertical map is the diagonal $\Delta_{\mathbf{Hilb}_A}$. (This formula is taken from Milne's notes on algebraic groups [MilneNotes, §9c].)

Then the functor of points point of view immediately shows that $\operatorname{Stab}_A(X)$ is a subgroup functor of $A$, which is a closed subgroup scheme since $\Delta_{\mathbf{Hilb}_A}$ is a closed immersion (see also the remark below). It is smooth since every algebraic group in characteristic $0$ is smooth [MilneNotes, Cor. 10.36] (but a better proof is given in Tag [047N](https://stacks.math.columbia.edu/tag/047N)), hence its identity component is an abelian subvariety of $A$ [MilneNotes, Def. 10.13].

Since formation of Hilbert schemes and the pullback square above commute with base change along $\operatorname{Spec} \bar K \to \operatorname{Spec} K$ (see [FGAE, 5.1.5(5)] for the statement about Hilbert schemes), we get
$$\operatorname{Stab}_{A_{\bar K}}(X_{\bar K}) = \operatorname{Stab}_A(X) \underset{\operatorname{Spec} K}\times \operatorname{Spec} \bar K.$$
The Nullstellensatz says that reduced subschemes of $A_{\bar K}$ are uniquely determined by their $\bar K$-points, so $\operatorname{Stab}_{A_{\bar K}}(X_{\bar K})$ could be defined simply as the (set-theoretic) stabiliser of $X_{\bar K}$ in $A_{\bar K}$.

**Remark.** It's a little dissatisfying that we need properness of $A$ to get representability of the Hilbert scheme, so the same argument doesn't work for affine algebraic groups. But in fact we don't really use representability of $\mathbf{Hilb}_A$, only that the diagonal $\Delta_{\mathbf{Hilb}_A}$ is representable by closed immersions (in the sense of Tags [0023](https://stacks.math.columbia.edu/tag/0023) and [025V](https://stacks.math.columbia.edu/tag/025V)). It's possible that this holds for a larger class of algebraic groups, but I don't immediately see whether or not this is true.

---

**References.**

[FGAE] <cite authors="Fantechi, Barbara; Göttsche, Lothar; Illusie, Luc; Kleiman, Steven L.; Nitsure, Nitin; Vistoli, Angelo"> B. Fantechi, L. Göttsche, L. Illusie, S. L. Kleiman, N. Nitsure, and A. Vistoli, [*Fundamental algebraic geometry: Grothendieck’s FGA explained*](https://doi.org/http://dx.doi.org/10.1090/surv/123). Mathematical Surveys and Monographs 123. American Mathematical Society (AMS), Providence, RI (2005).</cite>

[Hart] <cite authors="Hartshorne, Robin"> R. Hartshorne, [*Algebraic geometry*](https://doi.org/10.1007/978-1-4757-3849-0). Graduate Texts in Mathematics 52. Springer-Verlag, New York-Heidelberg-Berlin (1983).</cite>

[Milne] <cite authors="Milne, J. S."> J. S. Milne, [*Algebraic groups. The theory of group schemes of finite type over a field*](http://dx.doi.org/10.1017/9781316711736). Cambridge Studies in Advanced Mathematics 170. Cambridge University Press (2017).</cite>

A variant is available online as lecture notes (which I cite because I don't have the book available to me at the moment):

[MilneNotes] J. S. Milne, [*Algebraic groups*](https://www.jmilne.org/math/CourseNotes/iAG200.pdf). Lecture notes.