I think what you have defined is just called the category of endomorphisms in $\mathcal{C}$. See for instance Marian Mrozek's 1992 paper Normal functors and retractors in categories of endomorphisms for some properties of this category.
The oldest mention that I have found in the literature so far is
Almkvist, Gert, The Grothendieck ring of the category of endomorphisms, J. of Algebra 28 (1974), 375–388.
Almkvist restricts attention to the case where $\mathcal{C}$ is the category of $A$-modules which admit a finite projective resolution. See also this question of mine about homotopy properties of iterating the endomorphism construction.