There are at least two things that can go wrong. To explain this, let's carefully prove the following lemma.
Lemma. Let $X$ be a finite type $k$-scheme, let $k \to \ell$ be a finite extension, and let $\mathscr F$ be a coherent sheaf on $X_\ell$. Let $p \colon X_\ell \to X$ and $q \colon X_{k^{\operatorname{sep}}} \to X$ be the natural maps. Then
$$q^* p_* \mathscr F \cong \bigoplus_{\sigma \in \operatorname{Hom}_k(\ell,k^{\operatorname{sep}})} \sigma^* \mathscr F.$$
If $k \to \ell$ is Galois, then
$$p^*p_* \mathscr F \cong \bigoplus_{\sigma \in \operatorname{Gal}(\ell/k)} \mathscr F^{\sigma},$$
where $\mathscr F^{\sigma}$ is shorthand for $\sigma^* \mathscr F$.
Proof. By flat base change, we may replace $q^* p_*$ by $p'_* q'^*$, where $p'$ and $q'$ are the maps
$$\begin{array}{ccc}X_\ell \underset{X}\times X_{k^{\operatorname{sep}}} & \stackrel{p'}\to & \ X_{k^{\operatorname{sep}}} \\ {\scriptsize q'}\downarrow\ \ \ \ & & \downarrow {\scriptsize q} \\ X_\ell & \stackrel p\to & X. \ \end{array}$$
But $X_\ell \times_X X_{k^{\operatorname{sep}}}$ is none other than $X_{\ell \otimes_k k^{\operatorname{sep}}}$. We have a natural isomorphism
\begin{align*}
\ell \otimes_k k^{\operatorname{sep}} &\stackrel\sim\longrightarrow \prod_{\sigma \in \operatorname{Hom}_k(\ell, k^{\operatorname{sep}})} k^{\operatorname{sep}}\\
x \otimes y &\longmapsto \sigma(x)y.
\end{align*}
On the level of spectra, this means that $\operatorname{Spec}(\ell \otimes_k k^{\operatorname{sep}}) = \coprod_\sigma \operatorname{Spec} k^{\operatorname{sep}}$, with the natural map
$$\coprod_\sigma \operatorname{Spec} k^{\operatorname{sep}} = \operatorname{Spec}(\ell \otimes_k k^{\operatorname{sep}}) \to \operatorname{Spec} \ell$$
given on the component corresponding to $\sigma$ by $\sigma \colon \ell \to k^{\operatorname{sep}}$. Thus, $q'^* \mathscr F$ is given on the component corresponding to $\sigma$ by $\sigma^* \mathscr F$. This proves the first statement. The proof of the second statement is similar (but we should be careful about the two maps $p_1', p_2' \colon X_{\ell \otimes_k \ell} \to X_\ell$). $\square$
Remark. If $k \to \ell$ is a separable extension but possibly no longer Galois, then the primitive element theorem gives $\ell \cong k[x]/(f)$ for some irreducible polynomial $f \in k[x]$. Then we get
$$\ell \otimes_k \ell \cong \ell[x]/(f) \cong \ell \times \ell[x]/(g),$$
where $f = (x-\alpha)g$ for $\alpha \in \ell$ a root of $f$ and $g \in \ell[x]$. Thus, we always split off at least one factor $\ell$, which gives a summand of $p^*p_*\mathscr F$ isomorphic to $\mathscr F$ corresponding to the component $X_\ell$ of $X_{\ell \otimes_k \ell}$. So this does give you a map $\mathscr F \to p^*p_*\mathscr F$ splitting the counit $p^*p_*\mathscr F \to \mathscr F$ of the adjunction $p^* \vdash p_*$. This answers positively your second weaker question.
Remark. So what can go wrong for $p^* p_* \mathscr F$ to not be a power of $\mathscr F$?
Firstly, we can have a Galois extension $k \to \ell$ and a sheaf $\mathscr F$ such that $\mathscr F^\sigma$ is not isomorphic to $\mathscr F$ for some $\sigma \in \operatorname{Gal}(\ell/k)$. This is for example the case if $X = E$ is an elliptic curve over $\mathbb R$, and $\mathscr F = \mathcal O_{E_\mathbb C}(P)$ for a non-real point $P \in E(\mathbb C)$. But this is not rationally connected, so we will have to do better.
Another thing that can go wrong is that the splitting
$$q^* p_* \mathscr F \cong \bigoplus_{\sigma \in \operatorname{Hom}_k(\ell,k^{\operatorname{sep}})} \sigma^* \mathscr F.$$
is not defined over $\ell$ if $k \to \ell$ is not Galois.
It seems pretty hard to (provably) construct an example of the second type. Therefore, we will construct one of the first type, with $X_{\bar k}$ rationally connected.
Example. Let $X \subseteq \mathbb P^3_{\mathbb R}$ be the smooth quadric given by $x_0^2+x_1^2+x_2^2-x_3^2 = 0$. We have an isomorphism
\begin{align}
X_\mathbb C &\stackrel\sim\longrightarrow V(x_0x_3-x_1x_2) \subseteq \mathbb P^3_\mathbb C\label{Eq form}\tag{1}\\
[x_0:x_1:x_2:x_3] &\longmapsto [x_0+ix_1:x_2+x_3:x_3-x_2:x_0-ix_1].\nonumber
\end{align}
We also have an isomorphism (over any field)
\begin{align*}
\mathbb P^1 \times \mathbb P^1 &\stackrel\sim\longrightarrow V(x_0x_3-x_1x_2) \subseteq \mathbb P^3\\
([x_0:x_1],[y_0:y_1]) &\longmapsto [x_0y_0:x_0y_1:x_1y_0:x_1y_1].
\end{align*}
Under this isomorphism, the divisor $V(x_0) \in |\mathcal O_{\mathbb P^1 \times \mathbb P^1}(1,0)|$ corresponds to $V(x_0,x_1) \subseteq \mathbb P^3$. Under Galois conjugation in (\ref{Eq form}), this becomes $V(x_1,x_3)$, corresponding to $V(y_1) \in |\mathcal O(0,1)|$. Therefore, we conclude that $\sigma^* \mathcal O(1,0) = \mathcal O(0,1)$, where $\sigma \colon X_\mathbb C \to X_\mathbb C$ is the complex conjugation, and $X_\mathbb C$ is identified with $\mathbb P^1 \times \mathbb P^1$. Thus, by the analysis above, we have
$$p^*p_* \mathcal O(1,0) = \mathcal O(1,0) \oplus \mathcal O(0,1).$$
Thus, we have constructed a counterexample with $X_\bar k$ rationally connected and $\mathscr F$ locally free.
Closing remark. It seems that in general, it is hardly ever true that $p^*p_* \mathscr F \cong \mathscr F^{[\ell:k]}$, even if $X_{\bar k}$ is rationally connected and $\mathscr F$ is locally free.
