Let $C$ be a smooth curve of genus at least $3$. Then $C$ has a degree $1$ line bundle $L$ which is not effective, and by Riemann-Roch $$h^1(L) = -\chi(L) = g-2 > 0.$$ We have $$Ext^1(O_C,L) \cong H^1(L)\neq 0,$$ so we can let $E$ be a rank $2$ bundle given by a non-split extension $$0\to L \to E \to O_C\to 0.$$ This sequence destabilizes $E$, so $E$ is not semistable. But it is also simple. We want to compute $Hom(E,E)$, so we apply $Hom(E,-)$ to get $$0\to Hom(E,L)\to Hom(E,E)\to Hom(E,O_C).$$
To compute $Hom(E,L)$, we apply $Hom(-,L)$ to the original sequence and get $$0\to Hom(O_C,L) \to Hom(E,L) \to Hom(L,L)\to Ext^1(O_C,L).$$ Here $Hom(O_C,L) = H^0(L) = 0$ since $L$ is not effective, and the map $k\cdot id = Hom(L,L)\to Ext^1(O_C,L)$ is injective as it sends $id$ to the extension class defining $E$. Therefore $Hom(E,L) = 0$.
To compute $Hom(E,O_C)$, we apply $Hom(-,O_C)$ to the original sequence and get $$0\to Hom(O_C,O_C) \to Hom(E,O_C) \to Hom(L,O_C).$$ Here $Hom(L,O_C) = 0$ by degree considerations, and $Hom(O_C,O_C) = k$, so $Hom(E,O_C) = k$.
We conclude that $Hom(E,E) = k$.
I am not too aware of many sources that study much about simplicity. Stability or semistability is usually the correct notion to work with. There may be some happy accidents say on an elliptic curve where the notion of simplicity is better behaved.