Lemma. Let $k$ be a perfect field of characteristic $p > 0$, and let $X$ be a reduced $k$-variety. Then $H^i(X_{\operatorname{fppf}},\alpha_p)$ is a $k$-vector space. In particular, it can only be finite if it is $0$ or if $k$ is finite.
Proof. Consider the short exact sequence
\begin{equation}
0\to \alpha_p \to \mathbb G_a \stackrel{(-)^p}\to \mathbb G_a \to 0\tag{1}
\end{equation}
on $X_{\operatorname{fppf}}$. Let $f \colon X_{\operatorname{fppf}} \to X_{\operatorname{\acute et}}$ be the natural map. Then $R^if_* \mathbb G_a = 0$ for $i > 0$, since $\mathbb G_a$ is a smooth group scheme (see for example Milne's Étale Cohomology book, Theorem III.3.9). It follows that
$$R^if_* \alpha_p = 0$$
for $i \geq 2$. On the other hand, $f_* \alpha_p = 0$, since $X$ is reduced. Hence, the Leray spectral sequence
$$E_2^{p,q} = H^p(X_{\operatorname{\acute et}},R^qf_*(\alpha_p)) \Rightarrow H^{p+q}(X_{\operatorname{fppf}},\alpha_p)$$
collapses on the $E_2$ page, and we find
$$H^i(X_{\operatorname{fppf}},\alpha_p) = H^{i-1}(X_{\operatorname{\acute et}},R^1f_*\alpha_p).$$
But $R^1f_*\alpha_p$ sits in a short exact sequence
\begin{equation}
0 \to \mathcal O_X \stackrel{(-)^p}\to \mathcal O_X \to R^1f_*\alpha_p \to 0.\tag{2}
\end{equation}
If $F \colon X \to X$ denotes the absolute Frobenius, then $F$ is a homeomorphism on the underlying spaces. Moreover, $F_*\mathcal O_X \cong \mathcal O_X$ as abelian sheaves (even on $X_{\operatorname{\acute et}}$) (but not as sheaves of $\mathcal O_X$-modules!). Thus, we can rewrite (2) as
$$0 \to \mathcal O_X \stackrel {F^\#}\to F_*\mathcal O_X \to R^1f_* \alpha_p \to 0,$$
where the map $F^\# \colon \mathcal O_X \to F_* \mathcal O_X$ is just the map of rings corresponding to $F \colon X \to X$. This is an $\mathcal O_X$-linear map, so $R^1f_* \alpha_p$ is a coherent sheaf of $\mathcal O_X$-modules, thus its cohomology is a $k$-vector space. $\square$
Remark. Some texts use the relative Frobenius instead, but since $k$ is perfect I think it is harmless to identify $X$ with $X^{[p]}$, the pullback along the geometric Frobenius on $\operatorname{Spec} k$.
Remark. If $X$ is projective, then the same argument shows that $H^1(X,\alpha_p)$ is a finite-dimensional $k$-vector space.
If $X$ is projective and geometrically integral, then $H^0(X,\mathcal O_X) = k$. Since $k$ is perfect, (1) now gives an inclusion $H^1(X_{\operatorname{fppf}},\alpha_p) \subseteq H^1(X,\mathcal O_X)$.
On the other hand, if $X = \mathbb A^1$, then we get
$$H^1(X,\alpha_p) = k[x]/k[x]^p \cong \bigoplus\limits_{\substack{m \geq 0\\p \nmid m}} k \cdot x^m,$$
which is infinite-dimensional.
Remark. On the other hand, the short exact sequence on the flat site
$$0 \to \alpha_{p^n} \to \mathcal O_X \stackrel{F^n}\to \mathcal O_X \to 0$$
realises $H^1_{\operatorname{fppf}}(X, \alpha_{p^n})$ as the kernel of the map $H^1(X, \mathcal O_X) \to H^1(X, \mathcal O_X)$ induced by the $n$-th power of Frobenius.
We can think of $H^1(X,\mathcal O_X)$ as the $p$-Lie algebra of the Picard scheme, and I believe that the Frobenius on this space corresponds to the $p$-th power operation of the $p$-Lie algebra (Mumford's Abelian Varieties proves this when $X$ is an abelian variety [Thm 15.3], and I think the case for curves should follow by embedding into the Jacobian).
Relation with the $p$-rank. In [loc. cit.], Mumford shows that the $p$-rank $r$ is the dimension of the semisimple part of $H^1(X,\mathcal O_X)$ with respect to the Frobenius, and $g - r$ is the nilpotent part. It seems that therefore $g - r$ must equal the dimension of $H^1_{\operatorname{fppf}}(X, \alpha_{p^n})$ as $n \to \infty$; note that this is bounded since all are contained in $H^1(X, \mathcal O_X$).
Remark. A somewhat weird way of looking at $\alpha_{p^n}$: as above, if $f \colon X_{\operatorname{fppf}} \to X_{\operatorname{\acute et}}$ is the natural 'inclusion', then
$$R^if_* \alpha_{p^n} = \left\{\begin{array}{ll} \operatorname{coker}(\mathcal O_X \stackrel{F^n}\to (F_X^n)_* \mathcal O_X), & i = 1,\\ 0, & i \neq 1. \end{array}\right.$$
Here, the map $F^n$ is $\mathcal O_X$-linear. Thus, $R^1f_* \alpha_{p^n}$ is a coherent sheaf $\mathscr F$ of $\mathcal O_X$-modules, which can easily be seen to be locally free of rank $p^{n \dim X} - 1$ (if $X$ is smooth). Thus, we can think of (the sheaf) $\alpha_{p^n}$ as being like $\mathscr F[-1]$, at least in the sense that
$$H^i_{\operatorname{fppf}}(X, \alpha_{p^n}) = H^{i-1}(X, \mathscr F) = H^i(X, \mathscr F[-1]),$$
where the right hand side can be taken with respect to your favourite topology (Zariski, étale, fppf, etc.), since $\mathscr F$ is coherent.
However, $\alpha_{p^n}$ and $\mathscr F[-1]$ are not actually isomorphic in the derived category (consider sections on an infinitesimal thickening of $X$), but they become isomorphic after pushing forward to the étale site.
Remark. Happy new year!
