The category of abelian group objects Let $C$ be a category, say with finite products. What can be said about the category $\operatorname{Ab}(C)$ of abelian group objects of $C$? Is it always an abelian category? If not, what assumptions on $C$ have to be made? What happens when $C$ is the category of smooth proper geometrically integral schemes over some locally noetherian scheme $S$?
For example if $C=\mathrm{Set}$, we get of course the abelian category of abelian groups. If $C=\mathrm{Ring}/R$ for some ring $R$, then we get the abelian category $\operatorname{Mod}(R)$ (cf. nLab). In general, I have already trouble to show that $\operatorname{Hom}(A,B) \times \operatorname{Hom}(B,C) \to \operatorname{Hom}(A,C)$ is linear in the left coordinate if $A$, $B$, $C$ are abelian group objects.
 A: No, it fails badly in general. A simple example might be where $C = Top$: topological abelian groups do not form an abelian category. For example, this category is not balanced. 
In general one will need some exactness assumptions on $C$; I'll have to check into this carefully later (have to run now), but I think Barr-exactness of $C$ (regular, and equivalence relations are kernel pairs of their coequalizers) may be close to an ideal assumption. 
A: I'm not sure why this hasn't been mentioned yet, but the category of abelian group objects in smooth proper geometrically integral schemes is not abelian, already if $S = \operatorname{Spec} k$. Indeed, if $A$ is any abelian variety, then the map $[n] \colon A \to A$ for $n > 1$ is a monomorphism and an epimorphism:

*

*As the OP explained at What are the epimorphisms in the category of schemes?, it is an epimorphism of schemes since it is surjective on points and injective on sections (being a dominant morphism of integral schemes). Thus it is also an epimorphism of group schemes.

*If $f \colon G \to A$ is a map such that $G \to A \stackrel{[n]}\to A$ is trivial, then $f(G)$ lands in $A[n]$ scheme-theoretically. But if $G$ is geometrically integral, this implies $f(G)$ lands in the trivial subgroup $\boldsymbol 1$, again scheme-theoretically.

Thus, $[n] \colon A \to A$ is both a monomorphism and an epimorphism, but it clearly does not have an inverse (even as schemes) when $n > 1$.
Dropping "geometrically integral" doesn't help: if $\operatorname{char} k = p > 0$ and $A$ is supersingular, the same example with $n = p$ shows that looking at smooth abelian group schemes is not enough. Dropping both does work if you further impose finite type hypotheses: the category of abelian group schemes of finite type over a field $k$ is abelian; see Is the category of commutative group schemes abelian? and the references therein. Then the proper ones form a Serre subcategory, hence an abelian category.
It would be interesting to see what versions are true over a general base. It seems hard to me to make quotients if there are no flatness assumptions, but maybe kernels or cokernels of flat group schemes don't want to be flat. I should probably try to read some SGA3….
A: The abelian group objects in any elementary topos form an abelian category, see P.T. Johnstone's "Topos Theory", Theorem 8.11 (page 259).
A: Is $\mathscr{C}$ is regular/(exact in Barr sense) then for any algeraic theory $T$ the category $T{\operatorname{-Alg}}(\mathscr{C})$ of internal $T$-algebras is regular/(exact), in particular for $\mathscr{C}$ exact and $\operatorname{Ab} ={}$"commutative groups theory" we have  $\operatorname{Ab}(\mathscr{C})$ is exact, this is also additive (i.e. abelian, a category is abelian iff is additive and exact):
given $f, g: A\to B$ in $\operatorname{Ab}(\mathscr{C})$ get $f+g$ in either of the following two ways:

*

*appling the Yoneda valuation $h^X$, $X\in \mathscr{C}$ to $f, g: A\to B$ (and considering Yoneda's Lemma).


*$f+g: A \xrightarrow{(f, g)} B\times B \xrightarrow{+}B$
A: $\DeclareMathOperator\Ab{Ab}\DeclareMathOperator\Hom{Hom}$If $C$ is any additive category then every object has a unique structure as an abelian group object so $\Ab(C)=C$; but typically $C$ is not abelian.  For example, this applies to the category of free abelian groups.  One can also think about triangulated categories, which are usually not abelian, although a nice theorem of Freyd gives a canonical embedding in an abelian category.  One example that has been studied extensively is the category of spectra in the sense of stable homotopy theory.  Similarly, one can consider abelian group objects in the homotopy category of spaces, otherwise known as commutative H-spaces.
The question also says:

In general, I have already trouble to show that $\Hom(A,B)\times \Hom(B,C)\to \Hom(A,C)$ is linear in the left coordinate

Surely this is formal?  I have drawn the diagram but sadly I cannot get MathJax to display it.
Update:
Just to be clear about notation, I'll write $\mathcal{C}(X,Y)$ for morphism sets in $\mathcal{C}$, and $\Hom(A,B)$ for morphism sets in $Ab(\mathcal{C})$.  An object $A\in \Ab(\mathcal{C})$ has a natural abelian group structure on $\mathcal{C}(T,A)$  for all $T\in\mathcal{C}$.  Naturality means that $q\circ p+r\circ p=(q+r)\circ p$ for all $p:S\to T$ and $q,r:T\to A$.  Now let $B$ be another object of $\Ab(\mathcal{C})$.  A morphism in $\Ab(\mathcal{C})$ from $A$ to $B$ is just a morphism $f:A\to B$ in $\mathcal{C}$ with the property that $f\circ(p+q)=f\circ p+f\circ q$ for all $T$ and all $p,q\in\mathcal{C}(T,A)$.  Now suppose we have such morphisms $f,g:A\to B$ and $h,k\:B\to C$.  We then have
\begin{multline*} (f+g)\circ(p+q) = f\circ(p+q) + g\circ(p+q) = 
 f\circ p + g\circ p + f\circ q + g\circ q ={}
\\
(f+g)\circ p + (f+g)\circ q
\end{multline*}
(using the naturality of addition, the homomorphism property of $f$ and $g$, and then naturality again).  This shows that $f+g$ is again a homomorphism.  A similar argument shows that $h\circ f$, $h\circ g$ and $h\circ(f+g)$ are homomorphisms.  We have $h\circ(f+g)=h\circ f+h\circ g$ by the homomorphism property of $h$.  We also have $(h+k)\circ f=h\circ f+k\circ f$ by the naturality of addition.
