I do not think the concept of Moore-Penrose Inverse and the concept of categorical adjunction have much in common (except they both try to generalise the concept of inverse):

1. Equations $g = gfg$ and $fgf$ do not reaseble triangle equalities. Let me focus on the first equation. The "corresponding" triangle equality says that the composition $g \to^{\eta_g} gfg \to^{g\epsilon}g$ is the identity on $g$. Obviously $\eta_g$ is an inclusion, but generally there are no reasons for $\eta_g$ to be an isomorphism. However, in the world of 2-posets (i.e. categories enriched over posets) there is a reason. If we have a pair of 2-morphisms $\mathit{id} \le gf$ and $fg \le \mathit{id}$, then we may compose the first one on the right and the second on the left with $g$, obtaining $g \le gfg$ and $gfg \le g$, therefore $g = gfg$. Notice that I have not used any triangle equality here (just the existence of an appropriate pair of 2-morphisms).

2. The concept of an adjunction is inherently asymmetric --- a left adjoint is the best approximation of the identity from the left, and the right adjoint is the best approximation of the identity from the right; whilst the concept of Moore-Penrose pseudoinverse is perfectly symmetric. This means that if we had a 2-category, buit upon the category of vector spaces, where every pseudoinverse was a part of an adjunction (satisfying, perhaps, some other conditions), then every morphism would have both left and right adjoint, furthermore these adjoint functors would be isomorphic to the pseudoinverse (so isomorphic to each other). For a morphism $f$ in a 2-poset being both a left and right adjoint to $g$ is just being the inverse of $g$ --- the unit and counit from one adjunction give inequalities $\mathit{id} \le gf$ and $fg \le \mathit{id}$, whereas the unit and counit of the other adjunction give inequalities $\mathit{id} \le fg$ and $gf \le \mathit{id}$, hence $\mathit{id} = gf$ and $\mathit{id} = gf$. This fully answers your precise question, since in the category of vector space not every pesudoinverse is an inverse.

3. In any 2-category adjunctions compose --- if $f \colon A \to B \dashv f^+ \colon B \to A$ and $g \colon B \to C \dashv g^+ \colon C \to B$ then $g f \colon A \to C \dashv f^+ g^+ \colon C \to A$, but Moore-Penrose pseudoinverses --- generally --- do not. This almost answers your main question, because if $f \colon A \to B$, $g \colon B \to C$ are any maps between vector spaces then from the above property: $(gf)^+ \approx f^+ g^+$, so from this point of view pseudoinverses are not stable under isomorphisms, thus not categorical.

I do not think the concept of Moore-Penrose Inverse and the concept of categorical adjunction have much in common (except they both try to generalise the concept of inverse):

1. Equations $g = gfg$ and $fgf$ do not reaseble triangle equalities. Let me focus on the first equation. The "corresponding" triangle equality says that the composition $g \to^{\eta_g} gfg \to^{g\epsilon}g$ is the identity on $g$. Obviously $\eta_g$ is an inclusion, but generally there are no reasons for $\eta_g$ to be an isomorphism. However, in the world of 2-posets (i.e. categories enriched over posets) there is a reason. If we have a pair of 2-morphisms $\mathit{id} \le gf$ and $fg \le \mathit{id}$, then we may compose the first one on the right and the second on the left with $g$, obtaining $g \le gfg$ and $gfg \le g$, therefore $g = gfg$. Notice that I have not used any triangle equality here (just the existence of an appropriate pair of 2-morphisms).

2. The concept of an adjunction is inherently asymmetric --- a left adjoint is the best approximation of the identity from the left, and the right adjoint is the best approximation of the identity from the right; whilst the concept of Moore-Penrose pseudoinverse is perfectly symmetric. This means that if we had a 2-category, buit upon the category of vector spaces, where every pseudoinverse was a part of an adjunction (satisfying, perhaps, some other conditions), then every morphism would have both left and right adjoint, furthermore these adjoint functors would be isomorphic to the pseudoinverse (so isomorphic to each other). For a morphism $f$ in a 2-poset being both a left and right adjoint to $g$ is just being the inverse of $g$ --- the unit and counit from one adjunction give inequalities $\mathit{id} \le gf$ and $fg \le \mathit{id}$, whereas the unit and counit of the other adjunction give inequalities $\mathit{id} \le fg$ and $gf \le \mathit{id}$, hence $\mathit{id} = gf$ and $\mathit{id} = fg$. This fully answers your precise question, since in the category of vector spaces not every pesudoinverse is an inverse.

3. In any 2-category adjunctions compose --- if $f \colon A \to B \dashv f^+ \colon B \to A$ and $g \colon B \to C \dashv g^+ \colon C \to B$ then $g f \colon A \to C \dashv f^+ g^+ \colon C \to A$, but Moore-Penrose pseudoinverses --- generally --- do not. This almost answers your main question, because if $f \colon A \to B$, $g \colon B \to C$ are any maps between vector spaces then from the above property: $(gf)^+ \approx f^+ g^+$, so from this point of view pseudoinverses are not stable under isomorphisms, thus are not categorical.