Wojowu's idea is right:
Lemma. Let $R$ be a ring, let $\mathbf{Mod}_R$ be the category of (left) $R$-modules, and let $\mathbf{Mod}_R^{\text{fp}}$ be the subcategory of finitely presented modules. Then the following are equivalent:
- $R$ is left coherent, i.e. every finitely generated left ideal is finitely presented;
- $\mathbf{Mod}_R^{\text{fp}}$ is a weak Serre subcategory of $\mathbf{Mod}_R$;
- $\mathbf{Mod}_R^{\text{fp}}$ is abelian and the inclusion $\mathbf{Mod}_R^{\text{fp}} \hookrightarrow \mathbf{Mod}_R$ is exact;
- $\mathbf{Mod}_R^{\text{fp}}$ is abelian.
Proof. Implication 3 $\Rightarrow 4$ is obvious; implication 2 $\Rightarrow$ 3 is Tag 0754; and 1 $\Rightarrow$ 2 is well known (in the commutative case, see Tags 05CW and 05CX; the proofs generalise without difficulty to the general case).
For 4 $\Rightarrow$ 1, we note that the inclusion $\mathbf{Mod}_R^{\text{fp}} \hookrightarrow \mathbf{Mod}_R$ is left exact, as it is isomorphic to $\operatorname{Hom}_R(R,-)$. Now if $I \subseteq R$ is a finitely generated left ideal, then $M = R/I$ is finitely presented, i.e. $M \in \mathbf{Mod}_R^{\text{fp}}$. The morphism $R \to M$ is an epimorphism in $\mathbf{Mod}_R$, so in particular in $\mathbf{Mod}_R^{\text{fp}}$, so we get some short exact sequence
$$0 \to K \to R \to M \to 0$$
in $\mathbf{Mod}_R^{\text{fp}}$. By left exactness, we conclude that $K = I$ is the ideal we started with, which therefore has to be finitely presented. Thus $R$ is left coherent as $I \subseteq R$ was arbitrary. $\square$
(I feel there might be some general reason why $\mathbf{Mod}_R^{\text{fp}} \to \mathbf{Mod}_R$ satisfies nice properties, which could shorten the above proof.)
Example. Easy examples of coherent rings $R$ that are not Noetherian are valuation rings. Every finitely generated ideal of $R$ is principal, but $R$ is Noetherian if and only if the valuation is discrete.