Let me reduce the statement to one about fpqc descent. There should probably be an easier way to do this, but this is arguably the more 'correct' argument. The reduction of Galois descent for a finite extension $k \to \ell$ to fpqc descent is classical, and is for example recalled in [Stacks, Tag 0CDQ]. We merely adapt the argument to the infinite case.
The references I give are original (EGA and FGA), but a lot of this material is also available in the Stacks project's chapter on descent; see in particular [Stacks, Tag 08XA].
Lemma. Let $k \to \ell$ be a Galois extension with Galois group $G$. Then the natural map
\begin{align*}
\phi \colon \ell \underset k\otimes\ell &\to \prod_{\sigma \in G} \ell\\
a \otimes b &\mapsto \big(a \cdot \sigma(b)\big)_{\sigma\in G}
\end{align*}
is injective, and its image is
$$R = \bigg\{(x_\sigma)_\sigma\ \bigg|\ \exists U \subseteq G \text{ open subgroup}: x_{u\sigma} = x_\sigma \text{ for all } u \in U, \sigma \in G\bigg\}.$$
If this formula appears below, the assumption that $U \subseteq G$ be an open subgroup will be understood.
Proof. If $G$ is finite, then $\phi$ is an isomorphism. For example, write $\ell \cong k[x]/f$ for some monic irreducible polynomial $f \in k[x]$ that splits completely in $\ell$, and compute $\ell \otimes_k \ell$ as $\ell[x]/f$.
Thus, for $G$ profinite and $U \unlhd G$ an open normal subgroup, we get an isomorphism
\begin{align*}
\ell \underset k\otimes \ell^U &\stackrel\sim\to \prod_{\sigma \in G/U} \ell\tag{1}\label{Eq finite level}\\
a \otimes b &\mapsto (a \cdot\sigma(b))_{\sigma \in G/U}
\end{align*}
For any pair of open normal subgroups $U \subseteq V$ of $G$, there is a transition morphism
\begin{align*}
\prod_{\sigma \in G/V} \ell &\to \prod_{\sigma \in G/U} \ell\\
(x_\sigma)_{\sigma \in G/V} &\mapsto (x_{\pi(\sigma)})_{\sigma \in G/U},
\end{align*}
where $\pi \colon G/U \to G/V$ is the natural projection. These realise $R$ as the colimit
$$R = \underset{\ \ \longrightarrow\\ U \unlhd G\\\text{open}}{\operatorname{colim}} \prod_{\sigma \in G/U} \ell.$$
On the other hand, tensor products commute with colimits, so we have
$$\ell \underset k\otimes \ell \cong \underset{\ \ \longrightarrow\\ U \unlhd G\\\text{open}}{\operatorname{colim}} \ell \underset k\otimes \ell^U.$$
The result now follows from (\ref{Eq finite level}) by taking the colimits over all $U \unlhd G$ open and normal. $\square$
Remark 1. Following [Stacks, Tag 0CDQ], we write $f_\sigma \colon \ell \to \ell$ for the map $\sigma$. Given an $\ell$-module $N$, we write $f_\sigma^* N$ for the module $N \otimes_{\ell,\sigma} \ell$. Then an $\ell$-linear map $\phi \colon M \to f_\sigma^* N$ is the same thing as a map $\phi \colon M \to N$ such that $\phi(\lambda m) = \sigma(\lambda) \phi(m)$ for all $m \in M$, all $\lambda \in \ell$ (but only one choice of $\sigma$).
Thus, a semilinear action $G \times N \to N$ consists of isomorphisms $\phi_\sigma \colon N \to f_\sigma^* N$ satisfying the cocycle condition
$$\phi_{\sigma\tau} = f_\sigma^*\phi_\tau \circ \phi_\sigma\tag{2}\label{Eq cocycle}$$
for all $\sigma, \tau \in G$.
Remark 2. There are natural maps $\iota_0, \iota_1 \colon \ell \to \ell \otimes_k \ell$, and fpqc descent [FGA TDTE I, B, Thm. 1] tells us that a $k$-module $M$ is the same thing as an $\ell$-module $N$ together with an isomorphism of $\ell \otimes_k \ell$-modules
$$\phi \colon N \underset {\ell,\iota_0\!\!\!}\otimes \bigg( \ell \otimes_k \ell \bigg) \stackrel\sim\to N \underset {\ell,\iota_1\!\!\!}\otimes \bigg(\ell \otimes_k \ell\bigg)$$
satisfying a cocycle condition. As above, we will write $R$ for $\ell \otimes_k \ell$, and we use its description from the lemma as $R$. Then the two maps $\iota_i$ are given by
\begin{align*}
\iota_0 \colon \ell &\to R & \iota_1 \colon \ell &\to R \\
x &\mapsto (x)_\sigma, & x &\mapsto (\sigma(x))_\sigma.
\end{align*}
We also have projections $\pi_\sigma \colon R \to \ell$ given by $(x_\tau)_\tau \mapsto x_\sigma$, and $\pi_\sigma \circ \iota_0 = \operatorname{id}_\ell$, and $\pi_\sigma \circ \iota_1 = \sigma$.
Remark 3. If $N$ is an $\ell$-vector space, then
\begin{align*}
N \underset{\ell,\iota_0\!\!}\otimes R &\stackrel\sim\to \left\{ (n_\sigma)_\sigma \in \prod_{\sigma \in G} N\ \bigg|\ \exists U \subseteq G : n_{u\sigma} = n_\sigma \text{ for all }u \in U, \sigma \in G\right\}\\
n \otimes (x_\sigma)_\sigma &\mapsto (x_\sigma \cdot n)_\sigma.
\end{align*}
Indeed, the construction on the right hand side is given by colimits and finite products, both of which commute with arbitrary direct sums. Thus, the result for $N = \ell$ implies the general case. Similarly,
\begin{align*}
N \underset{\ell,\iota_1\!\!}\otimes R &\stackrel\sim\to \left\{ (n_\sigma)_\sigma \in \prod_{\sigma \in G} f_\sigma^*N\ \bigg|\ \exists U \subseteq G : n_{u \sigma} = f_u^*(n_\sigma) \text{ for all }u \in U, \sigma \in G\right\}\\
n \otimes (x_\sigma)_\sigma &\mapsto (x_\sigma \cdot f_\sigma^*(n))_\sigma.
\end{align*}
Remark 4. Thus, if $G \times N \to N$ is a $k$-semilinear action on an $\ell$-module $N$ such that all stabilisers are open, then Remark 1 gives $\ell$-linear isomorphisms $\phi_\sigma \colon N \to f_\sigma^* N$ satisfying the cocycle condition (\ref{Eq cocycle}). Using the description of Remark 3, we use this to define the descent datum
\begin{align*}
\phi \colon N \underset {\ell,\iota_0\!\!\!}\otimes R &\stackrel\sim\to N \underset {\ell,\iota_1\!\!\!}\otimes R\tag{3}\label{Eq descent datum}\\
(n_\sigma)_\sigma &\mapsto (\phi_\sigma(n_\sigma))_\sigma.
\end{align*}
Here, $\phi_\sigma(n_\sigma)$ satisfies the open subgroup condition precisely because $n_\sigma$ does and the action has open stabilisers: if $(n_\sigma)_\sigma \in N \otimes_{\ell,\iota_0} R$ is given, then there exist open subgroups $U, V \subseteq G$ such that $n_{u\sigma} = n_\sigma$ for all $u \in U$, all $\sigma \in G$ (see Remark 3), and $\phi_v(n_\sigma) = f_v^*(n_\sigma)$ for all $v \in V$, all $\sigma \in G$. Then for $\gamma \in U \cap V$, we have
\begin{align*}
\phi_{\gamma \sigma}(n_{\gamma \sigma}) &= f_\gamma^*(\phi_\sigma)(\phi_\gamma(n_{\gamma \sigma}))\tag{Cocycle condition}\\
&= f_\gamma^*(\phi_\sigma)(\phi_\gamma(n_\sigma))\tag{Property of $U$}\\
&= f_\gamma^*(\phi_\sigma)(f_\gamma^*(n_\sigma))\tag{Property of $V$}\\
&= f_\gamma^*(\phi_\sigma(n_\sigma)),
\end{align*}
showing the open subgroup property for $(\phi_\sigma(n_\sigma))_\sigma$. The cocycle condition (\ref{Eq cocycle}) implies that $\phi$ is a cocycle as well, hence a descent datum, hence effective by [FGA TDTE I, B, Thm. 1]. That is, if $M = N^G$, then the natural map $M \otimes_k \ell \to N$ is an isomorphism.
To get the statement for algebras instead of modules, we need to assume that the action $G \times N \to N$ is an algebra action. Then the descent isomorphism (\ref{Eq descent datum}) is an algebra isomorphism, so it is straightforward to see that this makes $M \otimes_k \ell \stackrel\sim\to N$ into an algebra isomorphism. $\square$
Final remark. As noted in the comments, the statement about finite generation of $M$ as an algebra if $N$ is a finitely generated $\ell$-algebra is [EGA IV$_2$, Lemma 2.7.1.1]. See also [Stacks, Tag 08XE].
References.
[EGA IV$_2$] Grothendieck, Alexander, Éléments de géométrie algébrique. IV: Étude locale des schémas et des morphismes de schémas. (Séconde partie), Publ. Math., Inst. Hautes Étud. Sci. 24, 1-231 (1965). ZBL0135.39701.
[FGA TDTE I] Grothendieck, Alexander, Technique de descente et théorèmes d’existence en géométrie algébrique. I: Généralités. Descente par morphismes fidèlement plats., Sem. Bourbaki 12 (1959/60), No. 190, 29 p. (1960). ZBL0229.14007.
[Stacks] A. J. de Jong et al, The stacks project.