Note, this answer was conceived before I understood Igor Belegradek's comments regarding the calculation of $p_1$.
Suppose $E$ is orientable, i.e. $w_1(E) = 0$. Then $w_2(\operatorname{End}(E)) = w_1(E)^2 = 0$, so $M$ must be spin.
Choosing an orientation for $E$, we can view $E$ as a complex line bundle. Then
$$\operatorname{End}(E) = \operatorname{End}_{\mathbb{C}}(E)\oplus\overline{\operatorname{End}}_{\mathbb{C}}(E)$$
where the terms of the decomposition are complex linear and complex antilinear endomorphisms respectively. If $J$ denotes the almost complex structure on $E$, then the decomposition is given by $L \mapsto \frac{1}{2}(L - JLJ) + \frac{1}{2}(L+JLJ)$. Note that $\operatorname{id}_E$ defines a nowhere-zero section of $\operatorname{End}_{\mathbb{C}}(E)$, so $\operatorname{End}_{\mathbb{C}}(E) \cong \varepsilon_{\mathbb{C}}^1$ (alternatively, $\operatorname{End}_{\mathbb{C}}(E) \cong E^*\otimes E \cong \varepsilon^1_{\mathbb{C}}$). On the other hand, a complex anti-linear endomorphism of $E$ can be viewed as a complex linear homomorphism $E \to \overline{E}$, so
$$\overline{\operatorname{End}}_{\mathbb{C}}(E) \cong \operatorname{Hom}_{\mathbb{C}}(E, \overline{E}) \cong E^*\otimes\overline{E} \cong \overline{E}^2.$$
Therefore
\begin{align*}
p_1(\operatorname{End}(E)) &= p_1(\varepsilon_{\mathbb{C}}^1\oplus\overline{E}^2)\\
&= p_1(\overline{E}^2)\\
&= -c_2(\overline{E}^2\otimes_{\mathbb{R}}\mathbb{C})\\
&= -c_2(\overline{E}^2\oplus E^2)\\
&= -c_1(\overline{E}^2)c_1(E^2)\\
&= -4c_1(\overline{E})c_1(E)\\
&= 4c_1(E)^2.
\end{align*}
Now we can use the Dold-Whitney Theorem.
Dold-Whitney Theorem: Oriented rank four real bundles on an closed, orientable four-manifold are uniquely determined by their second Stiefel-Whitney class $w_2$, Euler class $e$, and first Pontryagin class $p_1$.
Under the assumption that $E$ is orientable, the bundle $\operatorname{End}(E)$ is an oriented rank four real bundle on an orientable four-manifold with $w_2(\operatorname{End}(E)) = 0$, $e(\operatorname{End}(E)) = 0$ and $p_1(\operatorname{End}(E)) = 4c_1(E)^2$. If we can find an orientable four-manifold $M$ which is spin, has $\chi(M) = 0$, and $p_1(M) = 4c_1(E)^2$ for some complex line bundle $E$ on $M$, then by the Dold-Whitney Theorem, $TM \cong \operatorname{End}(E)$.
Let $M$ be a closed, smooth, spin four-manifold with $\chi(M) = 0$. If $\tau(M) = 0$, then $p_1(M) = 0 = 4c_1(E)^2$ where $E$ is the trivial complex line bundle. If $\tau(M) \neq 0$, then the intersection form of $M$ is of the form $\pm 2mE_8\oplus nH$ for some non-negative integers $m, n$. By Donaldson's theorem on definite intersection forms, if $m \neq 0$, then $n \neq 0$. So if $\tau(M) \neq 0$, it must have a copy of the hyperbolic lattice $H$ in its intersection form. Therefore every even integer is of the form $c^2$ for some $c \in H^2(M; \mathbb{Z})$. By Rokhlin's theorem, $\tau(M)$ is a multiple of $16$, so $\frac{3}{4}\tau(M) = \frac{1}{4}p_1(M)$ is a multiple of 12 (in particular, even), so there is a class $c$ such that $c^2 = \frac{1}{4}p_1(M)$, i.e. $p_1(M) = 4c^2$. As every element of $H^2(M; \mathbb{Z})$ is the first Chern class of some complex line bundle, we see that $p_1(M) = 4c_1(E)^2$ for some complex line bundle $E$. Therefore, we have the following:
Let $M$ be a closed, smooth, spin four-manifold with $\chi(M) = 0$. Then $TM \cong \operatorname{End}(E)$ for some $E$.
Moreover, by Dold-Whitney, such an $M$ is parallelisable if and only if $\tau(M) = 0$. In particular, if $M$ is spin with $\chi(M) = 0$, $\tau(M) \neq 0$, we get a non-trivial example.
Here's one way to construct such examples. Let $X$ be a closed, smooth, simply connected, spin four-manifold and set $M = X\# k(S^1\times S^3)$ where $k = \frac{1}{2}\chi(X)$. Then $M$ is a spin manifold with $\chi(M) = 0$, so $TM \cong \operatorname{End}(E)$ for some $E$. As $\tau(M) = \tau(X)$, we obtain non-trivial examples whenever $\tau(X) \neq 0$. The simplest non-parallelisable manifold that arises from this construction is
$$M = K3\#12(S^1\times S^3).$$
In this case $E$ is the unique complex line bundle with $c_1(E)^2 = -12$.